The status of the db2 pureScale instance is best determined by the db2instance -list and lssam commands.
For example:
db2psc@node02:~> db2instance -list
ID TYPE STATE HOME_HOST CURRENT_HOST ALERT
-- ---- ----- --------- ------------ -----
0 MEMBER STARTED node02 node02 NO
1 MEMBER STARTED node03 node03 YES
2 MEMBER STARTED node04 node04 YES
128 CF CATCHUP node02 node02 NO
129 CF PRIMARY node03 node03 NO
PARTITION_NUMBER LOGICAL_PORT NETNAME
---------------- ------------ -------
0 0 node02.purescale.ibm.local
0 0 node03.purescale.ibm.local
0 0 node04.purescale.ibm.local
- 0 node02.purescale.ibm.local
- 0 node03.purescale.ibm.local
HOSTNAME STATE INSTANCE_STOPPED ALERT
-------- ----- ---------------- -----
node04 ACTIVE NO YES
node03 ACTIVE NO NO
node02 ACTIVE NO NO
There is currently an alert for a member, CF, or host in the data-sharing instance.
For more information on the alert, its impact, and how to clear it,
run the following command: 'db2cluster -cm -list -alert'.
Look for the lssam output and it shows that one of the resource on node04 is failed offline.
db2psc@node02:~> lssam
Online IBM.ResourceGroup:ca_db2psc_0-rg Nominal=Online
'- Online IBM.Application:ca_db2psc_0-rs
|- Online IBM.Application:ca_db2psc_0-rs:node02
'- Online IBM.Application:ca_db2psc_0-rs:node03
Online IBM.ResourceGroup:db2_db2psc_0-rg Nominal=Online
'- Online IBM.Application:db2_db2psc_0-rs
|- Online IBM.Application:db2_db2psc_0-rs:node02
|- Offline IBM.Application:db2_db2psc_0-rs:node03
'- Offline IBM.Application:db2_db2psc_0-rs:node04
Online IBM.ResourceGroup:db2_db2psc_1-rg Control=MemberInProblemState Nominal=Online
'- Online IBM.Application:db2_db2psc_1-rs Control=MemberInProblemState
|- Offline IBM.Application:db2_db2psc_1-rs:node02
|- Online IBM.Application:db2_db2psc_1-rs:node03
'- Failed offline IBM.Application:db2_db2psc_1-rs:node04
When the lssam output shows Failed offline, it should trigger an adrenaline in DBA's mind, heart, body and soul that now it requires his/her attention to fix things.
We will try to force TSAMP to try to fix this. The resource db2_db2psc_1-rs has failed offline on node04 and we will try this command.
# resetrsrc -s 'Name == "db2_db2psc_1-rs" && NodeNameList="node04"' IBM.Application
The output from lssam shows that still that the node shows as failed offline and it is a good indication that something needs to be fixed in DB2 to get this state out of Failed offline.
Online IBM.ResourceGroup:db2_db2psc_1-rg Control=MemberInProblemState Nominal=Online
'- Online IBM.Application:db2_db2psc_1-rs Control=MemberInProblemState
|- Offline IBM.Application:db2_db2psc_1-rs:node02
|- Online IBM.Application:db2_db2psc_1-rs:node03
'- Failed offline IBM.Application:db2_db2psc_1-rs:node04
Normally the resetrsrc command can be used to try to fix things but if things are to be fixed at the DB2 level, the resource will be put back to the failed offline state again as seen above.
Let us look at the alerts that were thrown.
$ db2cluster -list -alert ------------------------------------------------------------------------------- 1. Alert: DB2 member '1' failed to successfully recover database 'psdb' during auto restart. Check the db2diag.log for message ADM7516N. Information prior to this message will provide more details on the failed database recovery. Action: Run the GET DB CONFIG command to determine if the database is in a backup pending, rollforward pending, or restore pending state and take appropriate action if the database is in a pending state. Run the RESTART DATABASE command to manually recover the failed database. The alert will automatically be cleared when the database is successfully recovered. Impact: Connections to the database will fail until the database has been successfully recovered. ------------------------------------------------------------------------------- 2. Alert: DB2 member '2' failed to successfully recover database 'psdb' during auto restart. Check the db2diag.log for message ADM7516N. Information prior to this message will provide more details on the failed database recovery. Action: Run the GET DB CONFIG command to determine if the database is in a backup pending, rollforward pending, or restore pending state and take appropriate action if the database is in a pending state. Run the RESTART DATABASE command to manually recover the failed database. The alert will automatically be cleared when the database is successfully recovered. Impact: Connections to the database will fail until the database has been successfully recovered. 3. Alert: Could not restart light DB2 member '1' on host 'node04'. Check the db2diag.log for messages concerning a restart light or database crash recovery failure on the indicated host for DB2 member '1'. Action: Check the cluster caching facility cfdiag log files for messages about CF failures on the host. If there are alerts about network adapters not responding, this alert cannot be cleared manually. It will be cleared when a network adapter becomes available. If it is not a problem with network adapters, this alert needs to be manually cleared after other alerts are handled. To clear this alert run the following command: 'db2cluster -cm -clear -alert -member 1'. For more information, see the 'Troubleshooting options for the db2cluster command' topic in the DB2 Information Center. Impact: DB2 member '1' will not be able to restart light on host 'node04' until this alert has been cleared.
The first alert is about the member 2 failed to successfully recover PSDB database. Look at the diagnostic log file for the node04 (member 2) to check.
The alert message says to look for ADM7516N and it says that the RESTART failed. But, it seems like a false alarm as the database is connectible.
There are 2 ways this failed offline can be fixed. The first is benign and second is malign. But, sometimes surgery is the only option.
Option – 1 : Try group restart (Reboot all nodes)
Option – 2 :
db2psc@node02:~> db2stop force 07/05/2012 10:53:44 1 0 SQL1064N DB2STOP processing was successful. 07/05/2012 10:53:45 2 0 SQL1064N DB2STOP processing was successful. 07/05/2012 10:53:45 0 0 SQL1064N DB2STOP processing was successful. db2psc@node02:~> su - Password: (Let your sysadmin type root password while you are pretending to look somewhere else) node02:~ # lsrpdomain Name OpState RSCTActiveVersion MixedVersions TSPort GSPort db2domain_20120703115447 Online 3.1.2.2 No 12347 12348 node02:~ # stoprpdomain -f db2domain_20120703115447 node02:~ # lssam lssam: No resource groups defined or cluster is offline! node02:~ # exit # cd /db2sd_<timestamp>/ db2psc@node02:~><instanceName>/sqllib_shared/ctrlha OR db2psc@node02:~> cd /db2sd_20120703115544/db2psc/sqllib_shared/ctrlha db2psc@node02:~> rm -f .*pgrp (Note .*pgrp - these are all hidden files) db2psc@node02:~> rm -f ../ctrlhamirror/.*pgrp exit Go to each DB2 member and delete .*pgrp from ~/sqllib/ctrlha and ~/sqllib/ctrlhamirror db2psc@node02:~> ssh node02 rm -f ~/sqllib/ctrlha/.*pgrp db2psc@node02:~> ssh node03 rm -f ~/sqllib/ctrlha/.*pgrp db2psc@node02:~> ssh node04 rm -f ~/sqllib/ctrlha/.*pgrp db2psc@node02:~> ssh node02 rm -f ~/sqllib/ctrlhamirror/.*pgrp db2psc@node02:~> ssh node03 rm -f ~/sqllib/ctrlhamirror/.*pgrp db2psc@node02:~> ssh node04 rm -f ~/sqllib/ctrlhamirror/.*pgrp Login as root. db2psc@node02:~> su - Password: node02:~ # lsrpdomain Name OpState RSCTActiveVersion MixedVersions TSPort GSPort db2domain_20120703115447 Offline 3.1.2.2 No 12347 12348 node02:~ # startrpdomain db2domain_20120703115447 Please wait for the domain to come online and it may take a little while. node02:~ # lsrpdomain 2610-412 A Resource Manager terminated while attempting to enumerate resources for this command. 2610-408 Resource selection could not be performed. node02:~ # lsrpdomain Name OpState RSCTActiveVersion MixedVersions TSPort GSPort db2domain_20120703115447 Online 3.1.2.2 No 12347 12348 Please wait for all nodes to come online. node02:~ # lsrpnode Name OpState RSCTVersion node03 Online 3.1.2.2 node02 Online 3.1.2.2 node04 Online 3.1.2.2 Please wait for all resources to start. Since we stopped db2, do not worry if you see red entries as Binding=Sacrificed or Control=StartInhibited or Request=Lock when you run lssam. node02:~ # lssam Pending online IBM.ResourceGroup:ca_db2psc_0-rg Binding=Sacrificed Nominal=Online '- Offline IBM.Application:ca_db2psc_0-rs Control=StartInhibited |- Offline IBM.Application:ca_db2psc_0-rs:node02 '- Offline IBM.Application:ca_db2psc_0-rs:node03 Pending online IBM.ResourceGroup:db2_db2psc_0-rg Request=Lock Nominal=Online '- Offline IBM.Application:db2_db2psc_0-rs Control=StartInhibitedBecauseSuspended |- Offline IBM.Application:db2_db2psc_0-rs:node02 |- Offline IBM.Application:db2_db2psc_0-rs:node03 '- Offline IBM.Application:db2_db2psc_0-rs:node04 Pending online IBM.ResourceGroup:db2_db2psc_1-rg Request=Lock Nominal=Online '- Offline IBM.Application:db2_db2psc_1-rs Control=StartInhibitedBecauseSuspended |- Offline IBM.Application:db2_db2psc_1-rs:node02 |- Offline IBM.Application:db2_db2psc_1-rs:node03 '- Offline IBM.Application:db2_db2psc_1-rs:node04 Pending online IBM.ResourceGroup:db2_db2psc_2-rg Request=Lock Nominal=Online '- Offline IBM.Application:db2_db2psc_2-rs Control=StartInhibitedBecauseSuspended |- Offline IBM.Application:db2_db2psc_2-rs:node02 |- Offline IBM.Application:db2_db2psc_2-rs:node03 '- Offline IBM.Application:db2_db2psc_2-rs:node04 Online IBM.ResourceGroup:db2mnt-db2data1-rg Nominal=Online '- Online IBM.Application:db2mnt-db2data1-rs |- Online IBM.Application:db2mnt-db2data1-rs:node02 |- Online IBM.Application:db2mnt-db2data1-rs:node03 '- Online IBM.Application:db2mnt-db2data1-rs:node04 Online IBM.ResourceGroup:db2mnt-db2data2-rg Nominal=Online '- Online IBM.Application:db2mnt-db2data2-rs |- Online IBM.Application:db2mnt-db2data2-rs:node02 |- Online IBM.Application:db2mnt-db2data2-rs:node03 '- Online IBM.Application:db2mnt-db2data2-rs:node04 Online IBM.ResourceGroup:db2mnt-db2data3-rg Nominal=Online '- Online IBM.Application:db2mnt-db2data3-rs |- Online IBM.Application:db2mnt-db2data3-rs:node02 |- Online IBM.Application:db2mnt-db2data3-rs:node03 '- Online IBM.Application:db2mnt-db2data3-rs:node04 Online IBM.ResourceGroup:db2mnt-db2data4-rg Nominal=Online '- Online IBM.Application:db2mnt-db2data4-rs |- Online IBM.Application:db2mnt-db2data4-rs:node02 |- Online IBM.Application:db2mnt-db2data4-rs:node03 '- Online IBM.Application:db2mnt-db2data4-rs:node04 Online IBM.ResourceGroup:db2mnt-db2sd_20120703115544-rg Nominal=Online '- Online IBM.Application:db2mnt-db2sd_20120703115544-rs |- Online IBM.Application:db2mnt-db2sd_20120703115544-rs:node02 |- Online IBM.Application:db2mnt-db2sd_20120703115544-rs:node03 '- Online IBM.Application:db2mnt-db2sd_20120703115544-rs:node04 Offline IBM.ResourceGroup:idle_db2psc_997_node02-rg Nominal=Offline '- Offline IBM.Application:idle_db2psc_997_node02-rs '- Offline IBM.Application:idle_db2psc_997_node02-rs:node02 Offline IBM.ResourceGroup:idle_db2psc_997_node03-rg Nominal=Offline '- Offline IBM.Application:idle_db2psc_997_node03-rs '- Offline IBM.Application:idle_db2psc_997_node03-rs:node03 Offline IBM.ResourceGroup:idle_db2psc_997_node04-rg Nominal=Offline '- Offline IBM.Application:idle_db2psc_997_node04-rs '- Offline IBM.Application:idle_db2psc_997_node04-rs:node04 Offline IBM.ResourceGroup:idle_db2psc_998_node02-rg Nominal=Offline '- Offline IBM.Application:idle_db2psc_998_node02-rs '- Offline IBM.Application:idle_db2psc_998_node02-rs:node02 Offline IBM.ResourceGroup:idle_db2psc_998_node03-rg Nominal=Offline '- Offline IBM.Application:idle_db2psc_998_node03-rs '- Offline IBM.Application:idle_db2psc_998_node03-rs:node03 Offline IBM.ResourceGroup:idle_db2psc_998_node04-rg Nominal=Offline '- Offline IBM.Application:idle_db2psc_998_node04-rs '- Offline IBM.Application:idle_db2psc_998_node04-rs:node04 Offline IBM.ResourceGroup:idle_db2psc_999_node02-rg Nominal=Offline '- Offline IBM.Application:idle_db2psc_999_node02-rs '- Offline IBM.Application:idle_db2psc_999_node02-rs:node02 Offline IBM.ResourceGroup:idle_db2psc_999_node03-rg Nominal=Offline '- Offline IBM.Application:idle_db2psc_999_node03-rs '- Offline IBM.Application:idle_db2psc_999_node03-rs:node03 Offline IBM.ResourceGroup:idle_db2psc_999_node04-rg Nominal=Offline '- Offline IBM.Application:idle_db2psc_999_node04-rs '- Offline IBM.Application:idle_db2psc_999_node04-rs:node04 Offline IBM.ResourceGroup:primary_db2psc_900-rg Nominal=Offline '- Offline IBM.Application:primary_db2psc_900-rs |- Offline IBM.Application:primary_db2psc_900-rs:node02 '- Offline IBM.Application:primary_db2psc_900-rs:node03 Online IBM.Equivalency:ca_db2psc_0-rg_group-equ |- Online IBM.PeerNode:node02:node02 '- Online IBM.PeerNode:node03:node03 Offline IBM.Equivalency:cacontrol_db2psc_equ |- Offline IBM.Application:cacontrol_db2psc_128_node02:node02 '- Offline IBM.Application:cacontrol_db2psc_129_node03:node03 Online IBM.Equivalency:db2_db2psc_0-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:db2_db2psc_1-rg_group-equ |- Online IBM.PeerNode:node03:node03 |- Online IBM.PeerNode:node04:node04 '- Online IBM.PeerNode:node02:node02 Online IBM.Equivalency:db2_db2psc_2-rg_group-equ |- Online IBM.PeerNode:node04:node04 |- Online IBM.PeerNode:node02:node02 '- Online IBM.PeerNode:node03:node03 Online IBM.Equivalency:db2_public_network_db2psc_0 |- Online IBM.NetworkInterface:eth0:node02 |- Online IBM.NetworkInterface:eth0:node03 '- Online IBM.NetworkInterface:eth0:node04 Online IBM.Equivalency:db2mnt-db2data1-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:db2mnt-db2data2-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:db2mnt-db2data3-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:db2mnt-db2data4-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:db2mnt-db2sd_20120703115544-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:idle_db2psc_997_node02-rg_group-equ '- Online IBM.PeerNode:node02:node02 Online IBM.Equivalency:idle_db2psc_997_node03-rg_group-equ '- Online IBM.PeerNode:node03:node03 Online IBM.Equivalency:idle_db2psc_997_node04-rg_group-equ '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:idle_db2psc_998_node02-rg_group-equ '- Online IBM.PeerNode:node02:node02 Online IBM.Equivalency:idle_db2psc_998_node03-rg_group-equ '- Online IBM.PeerNode:node03:node03 Online IBM.Equivalency:idle_db2psc_998_node04-rg_group-equ '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:idle_db2psc_999_node02-rg_group-equ '- Online IBM.PeerNode:node02:node02 Online IBM.Equivalency:idle_db2psc_999_node03-rg_group-equ '- Online IBM.PeerNode:node03:node03 Online IBM.Equivalency:idle_db2psc_999_node04-rg_group-equ '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:instancehost_db2psc-equ |- Online IBM.Application:instancehost_db2psc_node03:node03 |- Online IBM.Application:instancehost_db2psc_node02:node02 '- Online IBM.Application:instancehost_db2psc_node04:node04 Online IBM.Equivalency:primary_db2psc_900-rg_group-equ |- Online IBM.PeerNode:node02:node02 '- Online IBM.PeerNode:node03:node03 Verify resources node02:~ # exit db2psc@node02:~> db2cluster -cm -verify -resources Cluster manager resource states for the DB2 instance are consistent. If above shows that DB2 resources are inconsistent, run db2cluster -cm -repair -resources db2psc@node02:~> db2start 07/05/2012 11:14:27 0 0 SQL1063N DB2START processing was successful. 07/05/2012 11:14:29 1 0 SQL1063N DB2START processing was successful. 07/05/2012 11:14:31 2 0 SQL1063N DB2START processing was successful. Try lssam and confirm that there is no failed offline. db2psc@node02:~> lssam Online IBM.ResourceGroup:ca_db2psc_0-rg Nominal=Online '- Online IBM.Application:ca_db2psc_0-rs |- Online IBM.Application:ca_db2psc_0-rs:node02 '- Online IBM.Application:ca_db2psc_0-rs:node03 Online IBM.ResourceGroup:db2_db2psc_0-rg Nominal=Online '- Online IBM.Application:db2_db2psc_0-rs |- Online IBM.Application:db2_db2psc_0-rs:node02 |- Offline IBM.Application:db2_db2psc_0-rs:node03 '- Offline IBM.Application:db2_db2psc_0-rs:node04 Online IBM.ResourceGroup:db2_db2psc_1-rg Nominal=Online '- Online IBM.Application:db2_db2psc_1-rs |- Offline IBM.Application:db2_db2psc_1-rs:node02 |- Online IBM.Application:db2_db2psc_1-rs:node03 '- Offline IBM.Application:db2_db2psc_1-rs:node04 Online IBM.ResourceGroup:db2_db2psc_2-rg Nominal=Online '- Online IBM.Application:db2_db2psc_2-rs |- Offline IBM.Application:db2_db2psc_2-rs:node02 |- Offline IBM.Application:db2_db2psc_2-rs:node03 '- Online IBM.Application:db2_db2psc_2-rs:node04 Online IBM.ResourceGroup:db2mnt-db2data1-rg Nominal=Online '- Online IBM.Application:db2mnt-db2data1-rs |- Online IBM.Application:db2mnt-db2data1-rs:node02 |- Online IBM.Application:db2mnt-db2data1-rs:node03 '- Online IBM.Application:db2mnt-db2data1-rs:node04 Online IBM.ResourceGroup:db2mnt-db2data2-rg Nominal=Online '- Online IBM.Application:db2mnt-db2data2-rs |- Online IBM.Application:db2mnt-db2data2-rs:node02 |- Online IBM.Application:db2mnt-db2data2-rs:node03 '- Online IBM.Application:db2mnt-db2data2-rs:node04 Online IBM.ResourceGroup:db2mnt-db2data3-rg Nominal=Online '- Online IBM.Application:db2mnt-db2data3-rs |- Online IBM.Application:db2mnt-db2data3-rs:node02 |- Online IBM.Application:db2mnt-db2data3-rs:node03 '- Online IBM.Application:db2mnt-db2data3-rs:node04 Online IBM.ResourceGroup:db2mnt-db2data4-rg Nominal=Online '- Online IBM.Application:db2mnt-db2data4-rs |- Online IBM.Application:db2mnt-db2data4-rs:node02 |- Online IBM.Application:db2mnt-db2data4-rs:node03 '- Online IBM.Application:db2mnt-db2data4-rs:node04 Online IBM.ResourceGroup:db2mnt-db2sd_20120703115544-rg Nominal=Online '- Online IBM.Application:db2mnt-db2sd_20120703115544-rs |- Online IBM.Application:db2mnt-db2sd_20120703115544-rs:node02 |- Online IBM.Application:db2mnt-db2sd_20120703115544-rs:node03 '- Online IBM.Application:db2mnt-db2sd_20120703115544-rs:node04 Online IBM.ResourceGroup:idle_db2psc_997_node02-rg Nominal=Online '- Online IBM.Application:idle_db2psc_997_node02-rs '- Online IBM.Application:idle_db2psc_997_node02-rs:node02 Online IBM.ResourceGroup:idle_db2psc_997_node03-rg Nominal=Online '- Online IBM.Application:idle_db2psc_997_node03-rs '- Online IBM.Application:idle_db2psc_997_node03-rs:node03 Online IBM.ResourceGroup:idle_db2psc_997_node04-rg Nominal=Online '- Online IBM.Application:idle_db2psc_997_node04-rs '- Online IBM.Application:idle_db2psc_997_node04-rs:node04 Online IBM.ResourceGroup:idle_db2psc_998_node02-rg Nominal=Online '- Online IBM.Application:idle_db2psc_998_node02-rs '- Online IBM.Application:idle_db2psc_998_node02-rs:node02 Online IBM.ResourceGroup:idle_db2psc_998_node03-rg Nominal=Online '- Online IBM.Application:idle_db2psc_998_node03-rs '- Online IBM.Application:idle_db2psc_998_node03-rs:node03 Online IBM.ResourceGroup:idle_db2psc_998_node04-rg Nominal=Online '- Online IBM.Application:idle_db2psc_998_node04-rs '- Online IBM.Application:idle_db2psc_998_node04-rs:node04 Online IBM.ResourceGroup:idle_db2psc_999_node02-rg Nominal=Online '- Online IBM.Application:idle_db2psc_999_node02-rs '- Online IBM.Application:idle_db2psc_999_node02-rs:node02 Online IBM.ResourceGroup:idle_db2psc_999_node03-rg Nominal=Online '- Online IBM.Application:idle_db2psc_999_node03-rs '- Online IBM.Application:idle_db2psc_999_node03-rs:node03 Online IBM.ResourceGroup:idle_db2psc_999_node04-rg Nominal=Online '- Online IBM.Application:idle_db2psc_999_node04-rs '- Online IBM.Application:idle_db2psc_999_node04-rs:node04 Online IBM.ResourceGroup:primary_db2psc_900-rg Nominal=Online '- Online IBM.Application:primary_db2psc_900-rs |- Online IBM.Application:primary_db2psc_900-rs:node02 '- Offline IBM.Application:primary_db2psc_900-rs:node03 Online IBM.Equivalency:ca_db2psc_0-rg_group-equ |- Online IBM.PeerNode:node02:node02 '- Online IBM.PeerNode:node03:node03 Online IBM.Equivalency:cacontrol_db2psc_equ |- Online IBM.Application:cacontrol_db2psc_128_node02:node02 '- Online IBM.Application:cacontrol_db2psc_129_node03:node03 Online IBM.Equivalency:db2_db2psc_0-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:db2_db2psc_1-rg_group-equ |- Online IBM.PeerNode:node03:node03 |- Online IBM.PeerNode:node04:node04 '- Online IBM.PeerNode:node02:node02 Online IBM.Equivalency:db2_db2psc_2-rg_group-equ |- Online IBM.PeerNode:node04:node04 |- Online IBM.PeerNode:node02:node02 '- Online IBM.PeerNode:node03:node03 Online IBM.Equivalency:db2_public_network_db2psc_0 |- Online IBM.NetworkInterface:eth0:node02 |- Online IBM.NetworkInterface:eth0:node03 '- Online IBM.NetworkInterface:eth0:node04 Online IBM.Equivalency:db2mnt-db2data1-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:db2mnt-db2data2-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:db2mnt-db2data3-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:db2mnt-db2data4-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:db2mnt-db2sd_20120703115544-rg_group-equ |- Online IBM.PeerNode:node02:node02 |- Online IBM.PeerNode:node03:node03 '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:idle_db2psc_997_node02-rg_group-equ '- Online IBM.PeerNode:node02:node02 Online IBM.Equivalency:idle_db2psc_997_node03-rg_group-equ '- Online IBM.PeerNode:node03:node03 Online IBM.Equivalency:idle_db2psc_997_node04-rg_group-equ '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:idle_db2psc_998_node02-rg_group-equ '- Online IBM.PeerNode:node02:node02 Online IBM.Equivalency:idle_db2psc_998_node03-rg_group-equ '- Online IBM.PeerNode:node03:node03 Online IBM.Equivalency:idle_db2psc_998_node04-rg_group-equ '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:idle_db2psc_999_node02-rg_group-equ '- Online IBM.PeerNode:node02:node02 Online IBM.Equivalency:idle_db2psc_999_node03-rg_group-equ '- Online IBM.PeerNode:node03:node03 Online IBM.Equivalency:idle_db2psc_999_node04-rg_group-equ '- Online IBM.PeerNode:node04:node04 Online IBM.Equivalency:instancehost_db2psc-equ |- Online IBM.Application:instancehost_db2psc_node03:node03 |- Online IBM.Application:instancehost_db2psc_node02:node02 '- Online IBM.Application:instancehost_db2psc_node04:node04 Online IBM.Equivalency:primary_db2psc_900-rg_group-equ |- Online IBM.PeerNode:node02:node02 '- Online IBM.PeerNode:node03:node03
It should be a very rare case to do this when you could not find a way to remove Failed Offline attribute for a particular type of resource. Use this with caution and do it at your own risk. The automation to do this is in this script.