【翻译自mos文章】Oracle Cluster failed to start with ASM instance getting ORA-00443 ---改动/etc/resolv.conf所致

 

来源于：

Oracle Cluster failed to start with ASM instance getting ORA-00443 (文档 ID 2000868.1)

 

 

APPLIES TO:

Oracle Database - Enterprise Edition - Version 11.2.0.3 and later
Information in this document applies to any platform.

SYMPTOMS

Cluster was not starting on one node:

# ./crsctl start clusterCRS-2672: Attempting to start 'ora.asm' on 'tcpepsd2'CRS-5017: The resource action "ora.asm start" encountered the following error:ORA-00443: background process "LMD0" did not start. For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/log/tcpepsd2/agent/ohasd/oraagent_grid/oraagent_grid.log".CRS-2674: Start of 'ora.asm' on 'tcpepsd2' failedCRS-2679: Attempting to clean 'ora.asm' on 'tcpepsd2'CRS-2681: Clean of 'ora.asm' on 'tcpepsd2' succeededCRS-4000: Command Start failed, or completed with errors.

oraagent_grid.log=================2015-04-09 12:40:05.848: [ora.asm][36] {0:0:109} [start] InstAgent::start exception }2015-04-09 12:40:05.849: [   AGENT][36] {0:0:109} UserErrorException: Locale is2015-04-09 12:40:05.849: [ora.asm][36] {0:0:109} [start] clsnUtils::error Exception type=2 string=CRS-5017: The resource action "ora.asm start" encountered the following error:ORA-00443: background process "LMD0" did not start. For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/log/tcpepsd2/agent/ohasd/oraagent_grid/oraagent_grid.log".2015-04-09 12:40:05.849: [    AGFW][36] {0:0:109} sending status msg [CRS-5017: The resource action "ora.asm start" encountered the following error:ORA-00443: background process "LMD0" did not start. For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/log/tcpepsd2/agent/ohasd/oraagent_grid/oraagent_grid.log".] for start for resource: ora.asm 1 12015-04-09 12:40:05.849: [ora.asm][36] {0:0:109} [start] (:CLSN00107:) clsn_agent::start }

 

Tried starting the ASM instance manually but failed:

$ sqlplus / as sysasmSQL*Plus: Release 11.2.0.3.0 Production on Wed Apr 15 11:37:47 2015Copyright (c) 1982, 2011, Oracle. All rights reserved.Connected to an idle instance.SQL> startupORA-00443: background process "LMD0" did not startalert_+ASM2.log===============Fri Apr 17 11:45:33 2015NOTE: No asm libraries found in the systemMEMORY_TARGET defaulting to 285212672.* instance_number obtained from CSS = 2, checking for the existence of node 0...* node 0 does not exist. instance_number = 2Starting ORACLE instance (normal)--------------------Fri Apr 17 11:50:55 2015LMON started with pid=9, OS id=5163Fri Apr 17 11:52:27 2015Process LMD0 died, see its trace file <<<<<USER (ospid: 4932): terminating the instance due to error 443Instance terminated by USER, pid = 4932

+ASM2_lmd0_5165.trc===================*** 2015-04-17 11:50:55.846Async driver not configured : errno=13kjmdmi: pmon timed out in attaching.*** 2015-04-17 11:51:55.899Process diagnostic dump for oracle@TCPEPSD2 (PMON), OS id=5149,pid: 2, proc_ser: 1, sid: 1, sess_ser: 1 -------------------------------------------------------------------------------os thread scheduling delay history: (sampling every 1.000000 secs)  0.000000 secs at [ 11:51:55 ]  NOTE: scheduling delay has not been sampled for 0.380300 secs 0.001209 secs from [ 11:51:51 - 11:51:56 ], 5 sec avg  0.000346 secs from [ 11:50:56 - 11:51:56 ], 1 min avg  0.000340 secs from [ 11:50:55 - 11:51:56 ], 5 min avgloadavg : 0.03 0.03 0.07Swapinfo : Avail = 40154.05Mb Used = 14179.29MbSwap free = 25974.76Mb Kernel rsvd = 1118.85MbFree Mem = 6279.48Mb   F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME COMD1401 S grid 5149 1 0 154 20 e0000003ab571100 86537 e0000001c1ee2040 11:50:54 ? 0:00 asm_pmon_+ASM2Short stack dump: ksedsts()+544<-ksdxfstk()+48<-ksdxcb()+3216<-sspuser()+688<-<-_poll_sys()+48<-_poll()+224<-_res_send()+3072<-_res_query()+336<-_res_querydomain()+512<-_res_search()+1792<-C000000000BECA80<-_nss_dns_getipnodebyname()+64<-C000000000BE9AC0<-nss_search()+1056<-__getipnodebyname_r()+816<-_getipnodebyname()+352<-_getaddrinfo()+976<-snlinGetAddrInfo()+576<-nttbnd2addr()+704<-ntacbbnd2addr()+736<-ntacbnd2addr()+272<-nsc2addr()+464<-nscall1()+400<-nscall()+1952<-nsgrcOpen()+688<-nsgrDo()+96<-nsgrrg_Register()+448<-kmmlrl()+10528<-ksucln()+7424<-ksbrdp()+2736<-opirip()+1296<-opidrv()+1152<-sou2o()+256<-opimai_real()+352<-ssthrdmain()+576<-main()+336 

 

CAUSE

We can see OS syscalls in the above call stack:

poll()+224<-_res_send()+3072<-_res_query()+336<-_res_querydomain()+512<-_res_search()+1792<-C000000000BECA80<-_nss_dns_getipnodebyname()+64<-C000000000BE9AC0<-nss_search()+1056<-__getipnodebyname_r()+816<-_getipnodebyname()+352<-_getaddrinfo()

 

This means that the process is trying to resolve the host alias and is not able to complete that operation. Normally this suggest we may have an OS configuration issue in the DNS or LDAP configuration, if this is used. 

 

Checking further, found nslookup failed:

# nslookup TCPEPSD2*** Can't find server name for address 203.176.113.82: Timed out*** Can't find server name for address 203.176.113.84: Timed out*** Default servers are not availableUsing /etc/hosts on:  TCPEPSD2looking up FILESName:    TCPEPSD2Address:  10.30.2.120

 

SOLUTION

There were some changes made on /etc/resolve.conf. After removing them, crs has come up fine.