Reset RSCT Configuration (completely remove all cluster configuration)

# /usr/sbin/rsct/install/bin/recfgct

Change the tie-breaker

# chrsrc -c IBM.PeerNode OpQuorumTieBreaker='Operator'

Find out which tie-breaker is in use

# lsrsrc -c IBM.PeerNode OpQuorumTieBreaker

Which network adapter state is down (OpState = 2)

# lsrsrc -la IBM.NetworkInterface Name OpState NodeNameList IPAddress CommGroup

Deny quorum to a sub-domain before granting quorum to right sub-domain

# runact -c IBM.PeerDomain ResolveOpQuorumTie Ownership=0

Grant quorum to a sub-domain to resolve the tie

# runact -c IBM.PeerDomain ResolveOpQuorumTie Ownership=1

Check if automation is manual or set to Auto.

# lssamctrl
Automation Example
# lssamctrl
Displaying SAM Control information:
SAMControl:
 TimeOut = 60
 RetryCount = 3
 Automation = Auto
 ExcludedNodes = {}
 ResourceRestartTimeOut = 5
 ActiveVersion = [3.2.2.8,Tue 19 Apr 2016 11:32:38 PM EDT]
 EnablePublisher = Disabled
 TraceLevel = 31
 ActivePolicy = []
 CleanupList = {}
 PublisherList = {}

Set Automation to Manual

# samctrl -M T

Set Automation to Auto

# samctrl -M F

TSAMP will not issue a start or stop order against a resource while TSAMP is in manual mode. However, RSCT will still invoke the CritRsrcProtMethod to protect resources if its conditions are met.

Determine which resources are listed as critical

# lsrsrc -Ap -s 'Name like "%" & ResourceType=1' IBM.Application Name ProtectionMode
Critical Resources Example
# lsrsrc -Ap -s 'Name like "%" & ResourceType=1' IBM.Application Name ProtectionMode
Name ProtectionMode
Resource Persistent Attributes for IBM.Application
resource 1:
 Name = "db2_db2psc_1-rs"
 ProtectionMode = 1
resource 2:
 Name = "idle_db2psc_997_node03-rs"
 ProtectionMode = 1
resource 3:
 Name = "idle_db2psc_998_node03-rs"
 ProtectionMode = 1
resource 4:
 Name = "idle_db2psc_999_node03-rs"
 ProtectionMode = 1
resource 5:
 Name = "primary_db2psc_900-rs"
 ProtectionMode = 1
resource 6:
 Name = "db2_db2psc_0-rs"
 ProtectionMode = 1
resource 7:
 Name = "idle_db2psc_997_node02-rs"
 ProtectionMode = 1
resource 8:
 Name = "idle_db2psc_998_node02-rs"
 ProtectionMode = 1
resource 9:
 Name = "idle_db2psc_999_node02-rs"
 ProtectionMode = 1

The default value for ProtectionMode is 1. It is recommended that this values should be changed to a value of 3. The value of 1 is to the hard reset and reboot which can lead to data loss. Changing to 3 will do a sync, hard reset and reboot.

Displaying Resource Group request information

# lsrgreq -L -m

Determining Installed vs Active version information

# lssrc -ls IBM.RecoveryRM
IVN/AVN
# lssrc -ls IBM.RecoveryRM
Subsystem : IBM.RecoveryRM
PID : 105497
Cluster Name : db2domain_20160419233227
Node Number : 1
Daemon start time : 04/20/16 21:31:41
Daemon State: 
 My Node Name : node02
 Master Node Name : node03 (node number = 2)
 Our IVN : 3.2.2.8
 Our AVN : 3.2.2.8
 Our CVN : 0000011eab8c1b77 (0000011e)

List all defined relationships

# lsrel -Ab

List all defined equivalency

# lrequ -Ab

Stop domain without forced reboot

# lsrsrc -c IBM.PeerNode CritRsrcProtMethod
# chrsrc -c IBM.PeerNode CritRsrcProtMethod=5
# stoprpdomain -f <domain_name>

After you are done changing things and the domain is backup and running, set CritRsrcProtMethod value back to 3.

# chrsrc -c IBM.PeerNode CritRsrcProtMethod=5

Apply License and reload RSCT to pick it up

echo ####################################################
echo Reloading IBM.ConfigRMd daemon to pick-up license
echo ####################################################
PID=`ps -ef | grep IBM.RecoveryRM | grep -v grep | awk '{print $2}'`
if [ "$PID" != "" ] ; then
   echo Killing $PID of IBM.ConfigRMd to pick license info
   kill $PID
else
   echo PID not found
fi