通过GC创建dataguard备库失败一则

 

问题描述：

因为业务增长需求，需要在原来dataguard环境（一主两备）的基础上，新增一备库。

但通过grid control创建备库时失败，由于主库数据文件有100G左右，备份恢复到从库要半小时间左右（千兆网，50M/s）。

 

现象：

创建备库的作业失败

 

在主节点查看rman恢复日志，可用下面命令查看rman运行作业的日志

ps –ef|grep rman

会在/tmp目录下生成rman临时日志，可以看到数据库备份成功

 

在从节点查看日志：

[oracle@hotel07 trace]$ tail -100f alert_htdb7.log

Thu Jul 25 15:21:01 2013

Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST

Thu Jul 25 15:21:01 2013

alter database flashback on

Errors in file /u01/app/ora11g/diag/rdbms/htdb7/htdb7/trace/htdb7_ora_13427.trc:

ORA-38706:无法启用 FLASHBACK DATABASE 事件记录。

ORA-38788:需要更多的备用数据库恢复

ORA-38706 signalled during: alter database flashback on...

Thu Jul 25 15:21:03 2013

 

恢复数据时到此为此。

 

查看从库状态：

SQL> select open_mode from v$database;

 

OPEN_MODE

--------------------

MOUNTED

 

--尝试open数据库

SQL> alter database open;

alter database open

*

第 1行出现错误:

ORA-10458: standby database requires recovery

ORA-01152:文件 1 没有从过旧的备份中还原

ORA-01110:数据文件 1: '+DATA/htdb7/datafile/system.272.821719041'

 

--再查看日志

[oracle@hotel07 trace]$ tail -100f alert_htdb7.log

alter database open

Data Guard Broker initializing...

Data Guard Broker initialization complete

Signalling error 1152 for datafile 1!

Beginning standby crash recovery.

Serial Media Recovery started

Managed Standby Recovery starting Real Time Apply

Media Recovery Waiting for thread 1 sequence 27106

FAL[client]: Error fetching gap sequence, no FAL server specified

Thu Jul 25 15:35:33 2013

FAL[client]: Failed to request gap sequence

 GAP - SCN range: 0x000c.7af3d503 - 0x000c.7af3d503

 DBID 1083719948 branch 759079182

FAL[client]: All defined FAL servers have been attempted.

-------------------------------------------------------------

Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization

parameter is defined to a value that is sufficiently large

enough to maintain adequate log switch information to resolve

archivelog gaps.

-------------------------------------------------------------

Thu Jul 25 15:35:54 2013

Errors in file /u01/app/ora11g/diag/rdbms/htdb7/htdb7/trace/htdb7_m000_14915.trc:

ORA-01155:正在打开, 关闭,装载或卸装数据库

Thu Jul 25 15:36:23 2013

Standby crash recovery need archive log for thread 1 sequence 27106 to continue.

Please verify that primary database is transporting redo logs to the standby database.

Wait timeout: thread 1 sequence 27106

Standby crash recovery aborted due to error 16016.

Errors in file /u01/app/ora11g/diag/rdbms/htdb7/htdb7/trace/htdb7_ora_13883.trc:

ORA-16016:线程 1 sequence# 27106 的归档日志不可用

Recovery interrupted!

Completed standby crash recovery.

 

看上去是应该缺少归档日志，不能继续进行数据库恢复。

 

再查看从库的关键参数，发现并没有被修改。

 

解决办法：

所以尝试手工修改备库的参数，再进行open数据库。

1.修改数据库参数：

htdb4主:

alter system set log_archive_config='dg_config=(htdb4,htdb5,htdb6,htdb7)';

alter system set fal_server='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel06)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb6)(SERVER=DEDICATED)))','(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel05)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb5)(SERVER=DEDICATED)))','(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel07)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb7)(SERVER=DEDICATED)))';

alter system set log_archive_dest_4='service="(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel07)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb7)(SERVER=DEDICATED)))"','LGWR SYNC AFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="htdb7" net_timeout=30','valid_for=(all_logfiles,primary_role)';

alter system set log_archive_dest_state_4='ENABLE';

 

htdb7备:

alter system set log_archive_config='dg_config=(htdb4,htdb5,htdb6,htdb7)';

alter system set fal_server='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel06)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb6)(SERVER=DEDICATED)))','(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel05)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb5)(SERVER=DEDICATED)))','(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel04)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb4)(SERVER=DEDICATED)))';

alter system set log_archive_dest_2='service="(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel04)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb4)(SERVER=DEDICATED)))"','LGWR SYNC AFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="htdb4" net_timeout=30','valid_for=(all_logfiles,primary_role)';

 

其它备库：

htdb6:

alter system set log_archive_config='dg_config=(htdb4,htdb5,htdb6,htdb7)';

alter system set fal_server='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel04)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb4)(SERVER=DEDICATED)))','(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel05)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb5)(SERVER=DEDICATED)))','(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel07)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb7)(SERVER=DEDICATED)))';

 

htdb5:

alter system set log_archive_config='dg_config=(htdb4,htdb5,htdb6,htdb7)';

alter system set fal_server='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel04)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb4)(SERVER=DEDICATED)))','(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel06)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb6)(SERVER=DEDICATED)))','(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=hotel07)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=htdb7)(SERVER=DEDICATED)))';

 

2.再次打开数据库

--第一次还是报错

SQL> alter database open; 

alter database open

*

第 1行出现错误:

ORA-10458: standby database requires recovery

ORA-01152:文件 1 没有从过旧的备份中还原

ORA-01110:数据文件 1: '+DATA/htdb7/datafile/system.272.821719041'

 

--再次尝试打开成功

SQL> alter database open;

 

数据库已更改。

 

--查看状态

SQL> select open_mode from v$database;

 

OPEN_MODE

--------------------

READ ONLY

 

--此时查看日志

alter database open --第一次open

Data Guard Broker initializing...

Data Guard Broker initialization complete

Signalling error 1152 for datafile 1!

Beginning standby crash recovery.

Serial Media Recovery started

Managed Standby Recovery starting Real Time Apply

Media Recovery Waiting for thread 1 sequence 27106

Thu Jul 25 16:34:19 2013

RFS[1]: Assigned to RFS process 21182

RFS[1]: Opened log for thread 1 sequence 27107 dbid 1083719948 branch 759079182

Archived Log entry 1 added for thread 1 sequence 27107 rlc 759079182 ID 0x40a48484 dest 4:

Thu Jul 25 16:34:21 2013

Primary database is in MAXIMUM AVAILABILITY mode

Changing standby controlfile to RESYNCHRONIZATION level

Standby controlfile consistent with primary

RFS[2]: Assigned to RFS process 21187

RFS[2]: Selected log 7 for thread 1 sequence 27119 dbid 1083719948 branch 759079182

Thu Jul 25 16:34:21 2013

Standby crash recovery aborted due to error 16426.

Errors in file /u01/app/ora11g/diag/rdbms/htdb7/htdb7/trace/htdb7_ora_13883.trc:

ORA-16426:恢复操作请求了不正确的日志来应用重做数据。

Recovery interrupted!

Completed standby crash recovery.

Signalling error 1152 for datafile 1!

Errors in file /u01/app/ora11g/diag/rdbms/htdb7/htdb7/trace/htdb7_ora_13883.trc:

ORA-10458: standby database requires recovery

ORA-01152:文件 1 没有从过旧的备份中还原

ORA-01110:数据文件 1: '+DATA/htdb7/datafile/system.272.821719041'

ORA-10458 signalled during: alter database open...

Thu Jul 25 16:34:22 2013

RFS[3]: Assigned to RFS process 21194

RFS[3]: Selected log 8 for thread 1 sequence 27118 dbid 1083719948 branch 759079182

Thu Jul 25 16:34:23 2013

Archived Log entry 2 added for thread 1 sequence 27118 ID 0x40a48484 dest 1:

RFS[3]: Opened log for thread 1 sequence 27109 dbid 1083719948 branch 759079182

Thu Jul 25 16:34:23 2013

RFS[4]: Assigned to RFS process 21202

RFS[4]: Opened log for thread 1 sequence 27108 dbid 1083719948 branch 759079182

Thu Jul 25 16:34:23 2013

RFS[5]: Assigned to RFS process 21198

RFS[5]: Opened log for thread 1 sequence 27110 dbid 1083719948 branch 759079182

Archived Log entry 3 added for thread 1 sequence 27108 rlc 759079182 ID 0x40a48484 dest 4:

RFS[4]: Opened log for thread 1 sequence 27111 dbid 1083719948 branch 759079182

……

alter database open --第二次open时

Data Guard Broker initializing...

Data Guard Broker initialization complete

Signalling error 1152 for datafile 1!

Beginning standby crash recovery.

Serial Media Recovery started

Managed Standby Recovery starting Real Time Apply

Media Recovery Log +FRA/htdb7/archivelog/2013_07_25/thread_1_seq_27107.285.821723659

Media Recovery Log +FRA/htdb7/archivelog/2013_07_25/thread_1_seq_27108.288.821723663

Incomplete Recovery applied until change 53602571404 time 07/25/2013 15:20:25

Completed standby crash recovery.

Thu Jul 25 16:36:06 2013

SMON: enabling cache recovery

Dictionary check beginning

Thu Jul 25 16:36:06 2013

Errors in file /u01/app/ora11g/diag/rdbms/htdb7/htdb7/trace/htdb7_dbw0_13194.trc:

ORA-01186: ?? 201 ??????

ORA-01157: ????/?????? 201 - ??? DBWR ????

ORA-01110: ???? 201: '+DATA'

File 201 not verified due to error ORA-01157

Errors in file /u01/app/ora11g/diag/rdbms/htdb7/htdb7/trace/htdb7_dbw0_13194.trc:

ORA-01186: ?? 202 ??????

ORA-01157: ????/?????? 202 - ??? DBWR ????

ORA-01110: ???? 202: '+DATA'

File 202 not verified due to error ORA-01157

Dictionary check complete

Re-creating tempfile +DATA as +DATA/htdb7/tempfile/temp.304.821723767

Re-creating tempfile +DATA as +DATA/htdb7/tempfile/temp.305.821723767

Database Characterset is ZHS16GBK

No Resource Manager plan active

**********************************************************

WARNING: Files may exists in db_recovery_file_dest

that are not known to the database. Use the RMAN command

CATALOG RECOVERY AREA to re-catalog any such files.

Dictionary check complete

Re-creating tempfile +DATA as +DATA/htdb7/tempfile/temp.304.821723767

Re-creating tempfile +DATA as +DATA/htdb7/tempfile/temp.305.821723767

Database Characterset is ZHS16GBK

No Resource Manager plan active

**********************************************************

WARNING: Files may exists in db_recovery_file_dest

that are not known to the database. Use the RMAN command

CATALOG RECOVERY AREA to re-catalog any such files.

If files cannot be cataloged, then manually delete them

using OS command.

One of the following events caused this:

1. A backup controlfile was restored.

2. A standby controlfile was restored.

3. The controlfile was re-created.

4. db_recovery_file_dest had previously been enabled and

   then disabled.

**********************************************************

replication_dependency_tracking turned off (no async multimaster replication found)

Physical standby database opened for read only access.

Completed: alter database open

 

--查看dataguard环境环境正常

[oracle@hotel07 trace]$ dgmgrl sys/oracle123

DGMGRL for Linux: Version 11.2.0.2.0 - 64bit Production

 

Copyright (c) 2000, 2009, Oracle. All rights reserved.

 

欢迎使用 DGMGRL,要获取有关信息请键入 "help"。

已连接。

DGMGRL> show configuration

 

配置 - htdb4

 

 保护模式:        MaxAvailability

 数据库:

    htdb4 -主数据库

    htdb5 -物理备用数据库

    htdb6 -物理备用数据库

    htdb7 -物理备用数据库

 

快速启动故障转移: DISABLED

 

配置状态:

SUCCESS

 

分析原因：

可能是由于创建备库时，正是主库业务比较繁忙的时候，而且主库的数据文件比较大，备份恢复的时间又比较长，造成主库的部分日志没有传输到从节点，从而造成恢复失败。所在做dataguard时尽量在主库业务不忙时进行，确保安全和避免一些错误发生。