Put simply, when you install and configure an Management Agent in a clustered environment to monitor your targets through Enterprise Manager Cloud/Grid Control, then that primary node fails, your Oracle services are moved to a secondary node as part of the clustering…but your OMS isn’t aware of the change in nodes.
Now if your agent on the primary node is still running, this doesn’t cause too much of an issue as the agent could still monitor targets running on secondary node (although certain tasks via the EM Console will be limited).
If however your primary node has completely failed and/or the agent isn’t running, then you can use EMCLI (Enterprise Manager Command Line Interface) to tell your OMS that another agent should monitor the targets from now on by using the ‘relocate_targets’ command (more info here: Configuring Targets for Failover in Active/Passive Environments).
Assuming EMCLI has already been configured (see my blog about installing EMCLI in EM12c), then you simply use the following syntax:
./emcli relocate_targets -src_agent=[old_node]:[agent_port] -dest_agent=[new_node]:[agent_port] -target_name=[name] -target_type=[type] -copy_from_src -force=yes -ignoreTimeSkew=yes
For example, to move my TESTA database and its listener (LISTENER_TESTA):
./emcli relocate_targets -src_agent=server1.localdomain:3872 -dest_agent=server2.localdomain:3872 -target_name=TESTA -target_type=oracle_database -copy_from_src -force=yes -ignoreTimeSkew=yes
./emcli relocate_targets -src_agent=server1.localdomain:3872 -dest_agent=server2.localdomain:3872 -target_name=LISTENER_TESTA -target_type=oracle_listener -copy_from_src -force=yes -ignoreTimeSkew=yes
NOTE: The ‘-ignoreTimeSkew=yes’ parameter is optional here, but it will ignore time variations between the nodes, which would can otherwise cause the command to fail.
After each command you’ll see a message like this:
Moved all targets from server1.localdomain:3872 to server2.localdomain:3872
You can check it’s done what it says it’s done by querying your EM Repository Database if you wish:
COL target_name FOR A15 COL target_type FOR A20 COL host_name FOR A20 COL emd_url FOR A45 SELECT target_name, target_type, host_name, emd_url FROM sysman.mgmt_targets WHERE target_name LIKE '%TESTA%'; TARGET_NAME TARGET_TYPE HOST_NAME EMD_URL --------------- -------------------- -------------------- --------------------------------------------- TESTA oracle_database server2.localdomain https://server2.localdomain:3872/emd/main/ TESTA_sys oracle_dbsys LISTENER_TESTA oracle_listener server2.localdomain https://server2.localdomain:3872/emd/main/
If/when you want to move the targets back to their primary node, you just repeat the process above (obviously switching the node names).
Apparently this relocate_targets function works with Oracle Management Agents of 10.2.0.4 and above, although I’ve only tried this myself with 11g and 12c versions. Another thing to point out here, is that the function doesn’t work with, and “composite targets such as RAC Clusters and iAS targets” (see MOS 577443.1).
I’ll be posting another blog shortly which takes this process a step further, and I’m using a shell script to check if a target has been moved, and automate the relocate_targets commands required. Watch this space… 🙂