Cluster Database Targets and Database System Targets are in Pending Status in Oracle Enterprise Manager 12c and 13c

There are a couple of newly added cluster databases targets and database system targets are in “PENDING” status, though all the cluster database instances are showing up. EM_REPOS_SEV_EVAL dbms scheduler job in the EM repository database is responsible for calculating the status of composite targets such as cluster database, database system, group, EM service,Oracle Weblogic Domain,etc.

Below are the steps of how to resolve the “PENDING” issue of cluster databases.

1) Check internal repository jobs from OEM:

 Setup -> Manage Cloud Control -> Repository -> Repository Scheduler Jobs Status ->  Scroll down to “Composite Target Availability Computation job”

x

$which repvfy
/u01/app/oracle/product/Middleware1/oms/bin/repvfy 

$/u01/app/oracle/product/Middleware1/oms/bin/repvfy verify -level 9 -details
Please enter the SYSMAN password:
...
...
7. Orphaned running scheduler jobs
----------------------------------

JOB_NAME RUNNING_INSTANCE ELAPSED_TIME CPU_USED
---------------------------------------- ---------------- --------------------------------------------------------------------------- ---------------------------------------------------------------------------
EM_REPOS_SEV_EVAL +014 04:16:46.40

1 row selected.
...
..
.

2) Check DBMS_SCHEDULER is running and hanging for ages:

  • Go to the Homepage of the repository database
  • Go to  Administration/Oracle Scheduler/Jobs
  • Locate the EM_REPOS_SEV_EVAL job
  • Check the ‘scheduled date’,’last run date’ and ‘scheduled status

x

SYSMAN EM_FEATURE_USE_COLL TRUE
SYSMAN EM_PING_MARK_NODE_STATUS TRUE
SYSMAN EM_SLM_CALENDAR_UPDATE TRUE
SYSMAN EM_SLM_COMP_SCHED_JOB TRUE
SYSMAN EM_BEACON_GENSVC_AVAIL TRUE
SYSMAN USAGE_SUMMARY TRUE
SYSMAN EM_GROUP_MEMBER_SYNCUP TRUE
SYSMAN EM_AVAIL_UNKNOWN_STUCK TRUE
SYSMAN EM_AUDIT_EXT_SCHED_JOB TRUE
SYSMAN EM_SYSTEM_MEMBER_SYNUP TRUE
SYSMAN EM_BSLN_SET_THRESHOLDS TRUE
SYSMAN EM_BSLN_COMPUTE_STATS TRUE
SYSMAN EM_REPOS_SEV_EVAL TRUE
SYSMAN EM_HEALTH_CALC_JOB TRUE
SYSMAN EM_ALERTS_SYNC TRUE
SYSMAN EM_METBSLN_COMPUTE_STATS TRUE
SYSMAN EM_TASK_WORKER_6 TRUE
SYSMAN EM_TASK_WORKER_5 TRUE
SYSMAN EM_TASK_WORKER_4 TRUE
SYSMAN EM_TASK_WORKER_1 TRUE
SYSMAN EM_REPOS_SEV_EVAL FALSE
SYSMAN EM_JOBS_STEP_SCHED TRUE
SYSMAN EM_ECM_VCPU_JOB TRUE
SYSMAN EM_LM_TASK_JOB_MONITOR TRUE
SYSMAN EM_JOB_PURGE_POLICIES TRUE
SYSMAN EM_DERIV_RETRY_ACTIONS_JOB TRUE
SYSMAN EM_TGT_PROP_CONF_PP TRUE
SYSMAN EM_GATHER_SYSMAN_STATS TRUE
SYSMAN EM_PURGE_POLICIES TRUE
SYSMAN EM_DAILY_MAINTENANCE TRUE
SYSMAN EM_ROLLUP_SCHED_JOB TRUE
SYSMAN EM_TASK_RESUBMIT_FAILED TRUE
AUSPOST ARCHIVE_AUDITTRAIL TRUE

3)  Stop and restart the EM_REPOS_SEV_EVAL job. After a while, we will see the cluster database shows UP.

If above is not working, we need:

1) Stop the OMS

$<OMS_HOME>/bin>./emctl stop oms

2) set the database job_queue_processes parameter to zero.

SQL> show parameter job_queue_processes      ->>remember this value
SQL> alter system set job_queue_processes=0 scope=BOTH;

3)  Connect to the repository database as the SYSMAN user and run the following :

SQL>exec emd_maintenance.remove_em_dbms_jobs;
SQL> commit;

4) Reconnect to the repository database as the user with SYSDBA rights and reset the value of job_queue_processes to it’s original value

SQL>alter system set  job_queue_processes=<original value>  scope=BOTH;
For example:
SQL>alter system set job_queue_processes=1000 scope=BOTH;

5) Connect to the repository database as the ‘SYSMAN’ user and re-submit the DBMS_SCHEDULER jobs.

SQL>exec emd_maintenance.submit_em_dbms_jobs;
SQL>commit;

6) Start the OMS and re-check the repository jobs

$<OMS_HOME>/bin>./emctl start oms
Advertisements
This entry was posted in OEM, OEM12c and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s