The “Refresh From My Oracle Support” job in EM12c runs daily at 00:00 (midnight) by default. If the OMS host is under load at that time (backups etc.) then this can cause the refresh job to run past the 10 minutes StuckThreadMaxTime limit, which leads to a stuck thread being detected within WebLogic, and EM raising the following incident:
Host=[host]
Target type=Oracle WebLogic Server
Target name=/EMGC_GCDomain/GCDomain/EMGC_OMS1
Categories=Diagnostics, Fault
Message=Incident (BEA-337 [WebLogicServer]) detected in /u01/app/oracle/middleware/oms12c/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/GCDomain/EMGC_OMS1/alert/log.xml at time/line number: Wed Feb 6 00:10:17 2013/1289
Severity=Critical
Looking at the log.xml file mentioned above, you’ll see something like this which clearly shows the stuck thread:
</msg>
<msg time=’2013-02-06T00:10:17.356+00:00′ org_id=’oracle’ comp_id=’ofm’
msg_id=’185760012′ type=’INCIDENT_ERROR’ level=’1′
host_id='[host]’ host_addr=’147.188.215.34′ prob_key=‘BEA-337 [WebLogicServer]‘
upstream_comp=” downstream_comp=” ecid=’0000JmLQz4R8_quP4ar2Di1H2u2x000002′
errid=’118’ detail_path=‘/u01/app/oracle/middleware/oms12c/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/GCDomain/EMGC_OMS1/incident/incdir_118’>
<txt>Errors in directory: /u01/app/oracle/middleware/oms12c/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/GCDomain/EMGC_OMS1/incident/incdir_118 (incident=118):
stuck thread detected: [STUCK] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'
</txt>
</msg>
This can be resolved, by re-scheduling the “Refresh From My Oracle Support” job to a less busier time.
Make sure you log into the Cloud Control Console as SYSMAN
Navigate to > Enterprise > Job > Activity > select “Advanced Search” > select “Refresh From My Oracle Support” for the “Job Type” drop-down > “Go”
Then Edit the “Refresh From My Oracle Support” job and update the Schedule. I changed my Start Time from 12:00 AM to 12:00 PM whilst leaving the frequency etc. unchanged.
Hit Submit and that should do the trick!