After you install DB2 pureScale, it is a good idea to check the GPFS tie breaker and to see what DB2 install did to set the GPFS tie breaker. Run the following command.
# db2cluster -cfs -list -tiebreaker The current quorum device is of type Majority Node Set.
The above means that the surviving group of hosts with the majority will have an operational quorum and that is OK. Normally, you would see the above if you did the pureScale install using the command line and not by using db2setup command.
A GPFS may use several disks so how do you know which disk is used as a tie-breaker disk. You can run db2cluster -cfs -list -filesystem command to find out all file systems that are in use.
# db2cluster -cfs -list -filesystem FILE SYSTEM NAME MOUNT POINT --------------------------------- ------------------------- db2data1 /db2data/data1 db2data2 /db2data/data2 db2data3 /db2data/data3 db2data4 /db2data/data4 db2fs1 /db2fs1 db2logs /db2data/db2logs
And then, use db2cluster -cfs -list -filesystem <filesystemname>
Suppose, you want to designate one disk as a tie breaker for the GPFS, you will have to take an outage as it can not be done on the running system, which is not the case if you are setting up a tie breaker disk for the cluster manager aka RSCT or in plain terms Tivoli System Automation or TSA.
So, do a db2stop.
$ db2stop 01/28/2013 12:46:20 2 0 SQL1064N DB2STOP processing was successful. 01/28/2013 12:46:21 0 0 SQL1064N DB2STOP processing was successful. 01/28/2013 12:46:22 1 0 SQL1064N DB2STOP processing was successful. $ db2stop instance on node02 SQL1064N DB2STOP processing was successful. $ db2stop instance on node03 SQL1064N DB2STOP processing was successful. $ db2stop instance on node04 SQL1064N DB2STOP processing was successful.
Now, login as root and stop the cluster manager.
# db2cluster -cm -stop -host node02 All specified hosts have been stopped successfully. # lsrpnode lsrpnode: There are no nodes in the peer domain or an online peer domain does not exist. # lsrpdomain Name OpState RSCTActiveVersion MixedVersions TSPort GSPort db2domain_20130127144636 Offline 3.1.2.7 No 12347 12348
Stop GPFS
# db2cluster -cfs -stop -all
All specified hosts have been stopped successfully.
node02:~ # mmlsnode
GPFS nodeset Node list
------------- -------------------------------------------------------
db2cluster_20130127131658 node02 node03 node04
node02:~ # mmgetstate -a
Node number Node name GPFS state
------------------------------------------
1 node02 down
2 node03 down
3 node04 down
Set GPFS Tie Breaker
# db2cluster -cfs -set -tiebreaker -disk /dev/dm-1 The quorum type has been successfully changed to 'disk'. # db2cluster -cfs -list -tiebreaker The current quorum device is of type Disk with the following specifics: /dev/dm-1.
Start pureScale Cluster Manager
# lsrpdomain Name OpState RSCTActiveVersion MixedVersions TSPort GSPort db2domain_20130127144636 Offline 3.1.2.7 No 12347 12348 # db2cluster -cm -start -domain db2domain_20130127144636 lsrpdomain Name OpState RSCTActiveVersion MixedVersions TSPort GSPort db2domain_20130127144636 Online 3.1.2.7 No 12347 12348 # lsrpnode Name OpState RSCTVersion node04 Online 3.1.2.7 node02 Online 3.1.2.7 node03 Online 3.1.2.7
Make sure that all nodes are online and if one or more hosts are not online, check them.
Check if GPFS tie breaker is set or not?
# db2cluster -cfs -list -filesystem db2fs1 PATH ON LOCAL HOST OTHER KNOWN PATHS --------------------------------- ------------------------- (*) /dev/dm-1
The * sign against the disk /dev/dm-1 means that it is a tie breaker disk. The file system db2fs1 is associated with the disk /dev/dm-1.
Did you know that you can add as many disks as possible against a file system? You need to use rebalance option of the db2cluster -cfs to rebalance the disks. You can also remove a disk from the file system but you cannot remove that disk which shows an * sign against it. So, you have to change the tie breaker disk before you remove that disk.
Normally, you do not have to ever worry about the tie breaker in GPFS but you have to worry about the RSCT (or cluster manager) tie breaker disk. If the GPFS tie breaker is set to majority, there should not be any reason to change it. The above exercise is just an academic exercise to know how to set up a tie breaker in GPFS.