本帖最后由 snakestug 于 2018-1-3 22:49 编辑
最近一个月遇到两次ora-29740,导致关键业务中断,各位大神请帮忙分析下故障原因(怀疑是latch争用造成):
环境:三节点rac, oracle10.2.0.4.0 + AIX6.1
12月28日下午16:13分二号节点报ora-29740,随后该实例重启:
Thu Dec 28 16:12:47 2017
IPC Send timeout detected.Sender: ospid 4523878
Receiver: inst 1 binc 1008253767 ospid 10028908
Thu Dec 28 16:13:18 2017
IPC Send timeout to 0.15 inc 52 for msg type 65521 from opid 642
Thu Dec 28 16:13:18 2017
Communications reconfiguration: instance_number 1
Thu Dec 28 16:13:19 2017
Trace dumping is performing id=[cdmp_20171228161319]
Thu Dec 28 16:15:14 2017
Trace dumping is performing id=[cdmp_20171228161545]
Thu Dec 28 16:15:36 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmon_13173132.trc:
ORA-29740: evicted by member 1, group incarnation 54
Thu Dec 28 16:15:36 2017
LMON: terminating instance due to error 29740
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmsk_12583794.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lms5_12977144.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmsc_13631524.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmsh_12976180.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_pmon_5440552.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmsq_11666348.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmsl_11207884.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmst_10028216.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lms7_12715188.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmse_13304438.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lms9_11928694.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmsg_11470034.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lms8_12780708.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lms0_13566736.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lms6_2754540.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmsj_13174678.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmsv_12976528.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmso_10880116.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lmsf_13698642.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lms2_13633084.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:37 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g2_lms1_12453862.trc:
ORA-29740: evicted by member , group incarnation
Thu Dec 28 16:15:40 2017
Adjusting the default value of parameter parallel_max_servers
from 2560 to 1009 due to the value of parameter processes (1024)
Thu Dec 28 16:15:40 2017
Starting ORACLE instance (normal)
Thu Dec 28 16:15:41 2017
Shutting down instance (abort)
License high water mark = 762
一号节点:
Thu Dec 28 16:13:20 2017
IPC Send timeout detected. Receiver ospid 10028908
Receiver is waiting for a latch dumping latch state for receiver -17164
Thu Dec 28 16:13:20 2017
Errors in file /oracle/admin/ora10g/udump/ora10g1_ora_6226826.trc:
Thu Dec 28 16:13:50 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g1_lmse_10028908.trc:
Thu Dec 28 16:13:51 2017
Trace dumping is performing id=[cdmp_20171228161319]
Thu Dec 28 16:14:22 2017
Errors in file /oracle/admin/ora10g/bdump/ora10g1_lgwr_8520788.trc:
Thu Dec 28 16:14:23 2017
Errors in file /oracle/admin/ora10g/udump/ora10g1_ora_6947602.trc:
ORA-00600: internal error code, arguments: [4005], [0], [], [], [], [], [], []
Thu Dec 28 16:15:31 2017
Evicting instance 2 from cluster
Thu Dec 28 16:15:45 2017
Trace dumping is performing id=[cdmp_20171228161545]

2018-1-3 22:48 上传
点击文件名下载附件
2.05 MB, 下载次数: 0