OGG故障排除1例:因B机被修改数据导致replicat服务ABENDED修复

来源:互联网 发布:知己知彼软件好用吗 编辑:程序博客网 时间:2024/04/27 18:28
故障模拟如下:

A:HTogg01数据源,B:HTogg02备份

表AAA无主键,B机故意修改AAA一条记录,A机对AAA记录做修改,数据向B机同步时,导致B机OGG的 replicat abended。


[oracle@HTogg2 ~]$ cd /u01/ogg
[oracle@HTogg2 ogg]$ ./ggsci
GGSCI (HTogg2) 1> info all
Program     Status      Group       Lag at Chkpt  Time Since Chkpt
MANAGER     RUNNING                                           
REPLICAT    ABENDED     REPL01      00:00:07      01:10:24    
GGSCI (HTogg2) 2> info REPL01
REPLICAT   REPL01    Last Started 2016-06-28 11:18   Status ABENDED
Checkpoint Lag       00:00:07 (updated 01:11:06 ago)
Log Read Checkpoint  File ./dirdat/ht000006
                     2016-06-28 11:23:07.465387  RBA 1309
GGSCI (HTogg2) 3> sh more ggserr.log
...
2016-06-28 11:23:15  WARNING OGG-01004  Oracle GoldenGate Delivery for Oracle, repl01.prm:  Aborted grouped transaction on 'ITS.AAA', Database error 14
03 (OCI Error ORA-01403: no data found, SQL <UPDATE "ITS"."AAA" SET "A" = :a3,"N" = :a4,"DD" = :a5 WHERE "A" = :b0 AND "N" = :b1 AND "DD" = :b2 AND ROW
NUM = 1>).
2016-06-28 11:23:15  WARNING OGG-01003  Oracle GoldenGate Delivery for Oracle, repl01.prm:  Repositioning to rba 1309 in seqno 6.
2016-06-28 11:23:15  WARNING OGG-01154  Oracle GoldenGate Delivery for Oracle, repl01.prm:  SQL error 1403 mapping ITS.AAA to ITS.AAA OCI Error ORA-014
03: no data found, SQL <UPDATE "ITS"."AAA" SET "A" = :a3,"N" = :a4,"DD" = :a5 WHERE "A" = :b0 AND "N" = :b1 AND "DD" = :b2 AND ROWNUM = 1>.
2016-06-28 11:23:15  WARNING OGG-01003  Oracle GoldenGate Delivery for Oracle, repl01.prm:  Repositioning torba 1309 in seqno 6.
2016-06-28 11:23:15  ERROR   OGG-01296  Oracle GoldenGate Delivery for Oracle, repl01.prm:  Error mapping from ITS.AAA to ITS.AAA.
2016-06-28 11:23:15  ERROR   OGG-01668  Oracle GoldenGate Delivery for Oracle, repl01.prm:  PROCESS ABENDING.
...
GGSCI (HTogg2) 5> sh ls ./dirrpt -l
total 232
-rw-rw-rw- 1 oracle oinstall   2340 5月  11 15:28 MGR0.rpt
-rw-rw-rw- 1 oracle oinstall   2007 5月  11 14:20 MGR1.rpt
-rw-rw-rw- 1 oracle oinstall   2177 5月  11 14:19 MGR2.rpt
-rw-rw-rw- 1 oracle oinstall   2007 5月   5 15:13 MGR3.rpt
-rw-rw-rw- 1 oracle oinstall   2007 5月   5 14:43 MGR4.rpt
-rw-rw-rw- 1 oracle oinstall   2772 6月  28 11:18 MGR.rpt
-rw-rw-rw- 1 oracle oinstall  17984 5月  11 15:47 REPL010.rpt
-rw-rw-rw- 1 oracle oinstall  21105 5月  11 14:47 REPL011.rpt
-rw-rw-rw- 1 oracle oinstall 111035 5月  11 14:19 REPL012.rpt
-rw-rw-rw- 1 oracle oinstall  18000 5月   5 15:40 REPL013.rpt
-rw-rw-rw- 1 oracle oinstall   7198 5月   5 15:06 REPL014.rpt
-rw-rw-rw- 1 oracle oinstall   1442 6月  28 11:23 repl01.dsc
-rw-rw-rw- 1 oracle oinstall  16693 6月  28 11:23 REPL01.rpt
GGSCI (HTogg2) 17> sh more ./dirrpt/repl01.dsc
...
Current time: 2016-06-28 11:23:15
Discarded record from action ABEND on error 1403

OCI Error ORA-01403: no data found, SQL <UPDATE "ITS"."AAA" SET "A" = :a3,"N" =
:a4,"DD" = :a5 WHERE "A" = :b0 AND "N" = :b1 AND "DD" = :b2 AND ROWNUM = 1>
Aborting transaction on ./dirdat/ht beginning at seqno 6 rba 1309
                         error at seqno 6 rba 1309
Problem replicating ITS.AAA to ITS.AAA
Record not found
Mapping problem with compressed key update record (target format)...
*
A = aa
N = aa
DD = aa
A = aa
N = zz
DD = aa
*

Process Abending : 2016-06-28 11:23:15

[oracle@HTogg2 ogg]$ ./logdump

Oracle GoldenGate Log File Dump Utility for Oracle
Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230

Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved.
 
Logdump 29 >open ./dirdat/ht000006    【见:GGSCI (HTogg2) 2> info REPL01】
Current LogTrail is /u01/ogg/dirdat/ht000006
Logdump 30 >ghdr n
Error: Expected ON | OFF  
Logdump 31 >ghdr on
Logdump 32 >detail on
Logdump 33 >detail data
Logdump 34 >usertoken on
Logdump 35 >pos 1309
Reading forward from RBA 1309 【见:GGSCI (HTogg2) 2> info REPL01】
Logdump 36 >n
___________________________________________________________________
Hdr-Ind    :     E  (x45)     Partition  :     .  (x04)  
UndoFlag   :     .  (x00)     BeforeAfter:     A  (x41)  
RecLength  :    62  (x003e)   IO Time    : 2016/06/28 11:23:07.465.387   
IOType     :   115  (x73)     OrigNode   :   255  (xff)
TransInd   :     .  (x03)     FormatType :     R  (x52)
SyskeyLen  :     0  (x00)     Incomplete :     .  (x00)
AuditRBA   :          0       AuditPos   : 41041424
Continued  :     N  (x00)     RecCount   :     1  (x01)

2016/06/28 11:23:07.465.387 GGSPKUpdate          Len    62 RBA 1309
Name: ITS.AAA
After  Image:                                             Partition 4   G  s   
 001e 0000 0006 0000 0002 6161 0001 0006 0000 0002 | ..........aa........  
 6161 0002 0006 0000 0002 6161 0000 0006 0000 0002 | aa........aa........  
 6161 0001 0006 0000 0002 7a7a 0002 0006 0000 0002 | aa........zz........  
 6161                                              | aa  
Before Image          Len    32 (x00000020)
KeyLen    30 (x0000001e)
KeyCol     0 (x0000), Len     6 (x0006)  
 0000 0002 6161                                    | ....aa  
KeyCol     1 (x0001), Len     6 (x0006)  
 0000 0002 6161                                    | ....aa  
KeyCol     2 (x0002), Len     6 (x0006)  
 0000 0002 6161                                    | ....aa  

After Image           Len    30 (x0000001e)
Column     0 (x0000), Len     6 (x0006)  
 0000 0002 6161                                    | ....aa  
Column     1 (x0001), Len     6 (x0006)  
 0000 0002 7a7a                                    | ....zz  
Column     2 (x0002), Len     6 (x0006)  
 0000 0002 6161                                    | ....aa  
   
Logdump 38 >scanforendtrans
End of Transaction found at RBA 1479
___________________________________________________________________
Hdr-Ind    :     E  (x45)     Partition  :     .  (x04)  
UndoFlag   :     .  (x00)     BeforeAfter:     A  (x41)  
RecLength  :    27  (x001b)   IO Time    : 2016/06/28 12:35:33.471.319   
IOType     :     5  (x05)     OrigNode   :   255  (xff)
TransInd   :     .  (x03)     FormatType :     R  (x52)
SyskeyLen  :     0  (x00)     Incomplete :     .  (x00)
AuditRBA   :          9       AuditPos   : 9893392
Continued  :     N  (x00)     RecCount   :     1  (x01)

2016/06/28 12:35:33.471.319 Insert               Len    27 RBA 1479
Name: ITS.BBB
After  Image:                                             Partition 4   G  s   
 0000 0005 0000 0001 6100 0100 0500 0000 0162 0002 | ........a........b..  
 0005 0000 0001 63                                 | ......c  
Column     0 (x0000), Len     5 (x0005)  
 0000 0001 61                                      | ....a  
Column     1 (x0001), Len     5 (x0005)  
 0000 0001 62                                      | ....b  
Column     2 (x0002), Len     5 (x0005)  
 0000 0001 63                                      | ....c  
   
Logdump 39 >pos 1479
Reading forward from RBA 1479
Logdump 40 >n
___________________________________________________________________
Hdr-Ind    :     E  (x45)     Partition  :     .  (x04)  
UndoFlag   :     .  (x00)     BeforeAfter:     A  (x41)  
RecLength  :    27  (x001b)   IO Time    : 2016/06/28 12:35:33.471.319   
IOType     :     5  (x05)     OrigNode   :   255  (xff)
TransInd   :     .  (x03)     FormatType :     R  (x52)
SyskeyLen  :     0  (x00)     Incomplete :     .  (x00)
AuditRBA   :          9       AuditPos   : 9893392
Continued  :     N  (x00)     RecCount   :     1  (x01)

2016/06/28 12:35:33.471.319 Insert               Len    27 RBA 1479
Name: ITS.BBB
After  Image:                                             Partition 4   G  s   
 0000 0005 0000 0001 6100 0100 0500 0000 0162 0002 | ........a........b..  
 0005 0000 0001 63                                 | ......c  
Column     0 (x0000), Len     5 (x0005)  
 0000 0001 61                                      | ....a  
Column     1 (x0001), Len     5 (x0005)  
 0000 0001 62                                      | ....b  
Column     2 (x0002), Len     5 (x0005)  
 0000 0001 63                                      | ....c  
   
Logdump 41 >scanforheader
___________________________________________________________________
Hdr-Ind    :     E  (x45)     Partition  :     .  (x04)  
UndoFlag   :     .  (x00)     BeforeAfter:     A  (x41)  
RecLength  :    27  (x001b)   IO Time    : 2016/06/28 12:35:33.471.319   
IOType     :     5  (x05)     OrigNode   :   255  (xff)
TransInd   :     .  (x03)     FormatType :     R  (x52)
SyskeyLen  :     0  (x00)     Incomplete :     .  (x00)
AuditRBA   :          9       AuditPos   : 9893392
Continued  :     N  (x00)     RecCount   :     1  (x01)

2016/06/28 12:35:33.471.319 Insert               Len    27 RBA 1615
Name: ITS.BBB
After  Image:                                             Partition 4   G  s   
 0000 0005 0000 0001 6100 0100 0500 0000 0162 0002 | ........a........b..  
 0005 0000 0001 63                                 | ......c  
Column     0 (x0000), Len     5 (x0005)  
 0000 0001 61                                      | ....a  
Column     1 (x0001), Len     5 (x0005)  
 0000 0001 62                                      | ....b  
Column     2 (x0002), Len     5 (x0005)  
 0000 0001 63                                      | ....c  
   
Logdump 43 >quit
[oracle@HTogg2 ogg]$ ./ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Apr 23 2012 08:32:14

Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved.


减小步长由300修改为1:
GGSCI (HTogg2) 1> edit params repl01
    grouptransops 1
    maxtransops 1

GGSCI (HTogg2) 5> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
REPLICAT    ABENDED     REPL01      00:00:00      00:03:51    

[extseqno 见./dirrpt/repl01.dsc “Aborting transaction on ./dirdat/ht beginning at seqno 6 rba 1309”
extrba 见 Logdump 41 >scanforheader  “2016/06/28 12:35:33.471.319 Insert               Len    27 RBA 1615

]
GGSCI (HTogg2) 6> alter repl01,extseqno 6,extrba 1615
REPLICAT altered.


GGSCI (HTogg2) 7> start repl01

Sending START request to MANAGER ...
REPLICAT REPL01 starting


GGSCI (HTogg2) 8> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
REPLICAT    RUNNING     REPL01      00:14:57      00:00:00    

GGSCI (HTogg2) 9> stop repl01
GGSCI (HTogg2) 10> edit params repl01
    grouptransops 3000
    del->maxtransops 1
GGSCI (HTogg2) 11> start repl01

GGSCI (HTogg2) 12> info all
Program     Status      Group       Lag at Chkpt  Time Since Chkpt
MANAGER     RUNNING                                           
REPLICAT    RUNNING     REPL01      00:14:57      00:00:00    

GGSCI (HTogg2) 19> show repl01

Parameter settings:

SET SUBDIRS    ON
SET DEBUG      OFF

Current directory: /u01/ogg

Using subdirectories for all process files

Editor:  vi

Reports (.rpt)                 /u01/ogg/dirrpt
Parameters (.prm)              /u01/ogg/dirprm
Stdout (.out)                  /u01/ogg/dirout
Replicat Checkpoints (.cpr)    /u01/ogg/dirchk
Extract Checkpoints (.cpe)     /u01/ogg/dirchk
Process Status (.pcs)          /u01/ogg/dirpcs
SQL Scripts (.sql)             /u01/ogg/dirsql
Database Definitions (.def)    /u01/ogg/dirdef


GGSCI (HTogg2) 20> view params repl01

REPLICAT REPL01
SETENV (NLS_LANG="SIMPLIFIED CHINESE_CHINA.AL32UTF8")
USERID ogg,PASSWORD ogg
REPORT AT 01:50
REPORTCOUNT EVERY 30 MINUTES, RATE
REPORTROLLOVER AT 02:00
REPERROR DEFAULT, ABEND
NUMFILES 1000
GROUPTRANSOPS 3000
ASSUMETARGETDEFS
DISCARDFILE ./dirrpt/repl01.dsc, APPEND, MEGABYTES 200
DISCARDROLLOVER AT 02:00
GETTRUNCATES
ALLOWNOOPUPDATES
DDL INCLUDE ALL
DDLERROR DEFAULT IGNORE RETRYOP MAXRETRIES 3 RETRYDELAY 5
DDLOPTIONS REPORT
MAPEXCLUDE ITS.BAYMONRECSTAT
EOFDELAYCSECS 10
map its.its_seq, target its.its_seq;
MAP ITS.*,TARGET ITS.*;


GGSCI (HTogg2) 21> info repl01

REPLICAT   REPL01    Last Started 2016-06-28 13:50   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:06 ago)
Log Read Checkpoint  File ./dirdat/ht000006
                     First Record  RBA 1870




0 0
原创粉丝点击