DB2 10.1 pureScale requires an optional but highly recommended SCSI-3 PR capable storage.
DB2 10.1 pureScale requires a mandatory tie breaker disk which should be SCSI-3 PR capable. Optionally, you can configure an IP address to be a tie-breaker disk but this is not documented and can be configured using db2cluster command such as:
db2cluster -cm -set -tiebreaker -ip 192.168.142.2
However, the SCSI-3 PR capability regardless of the storage comes from the driver. This will be evident from the following:
On SLES 11 SP1, if we install iscsitarget and iscsitarget-kmp-default packages and use iscsitarget to do the poor man’s SAN using the disks from one server to the another, we do not get SCSI-3 capable storage.
You can check this through sg_persist command.
For example:
# sg_persist -d /dev/sdg --out --register --param-rk=0 --param-sark=0x00006d0000000001
The output may show such as
IET VIRTUAL_DISK 0 Peripheral device type: disk PR Out:, command not supported
However, on SLES 11 SP2, the iscsitarget driver has support for the SCSI-3 PR.
Step-1: Run the command to check the state of the disk:
# sg_persist -i -k /dev/sdh
IET VIRTUAL-DISK 0
Peripheral device type: disk
PR generation=0xf, 1 registered reservation key follows:
0xb4a377d435a980f3
If the above command shows that there are existing keys registered / reserved, clear those keys first using the following command before going to the next step. For example, key 0xb4a377d435a980f3 is the registered reservation on the disk.
# sg_persist -d /dev/sdh -o -C -K 0xb4a377d435a980f3 IET VIRTUAL-DISK 0 Peripheral device type: disk
Again check the state of the disk by using
# sg_persist -i -k /dev/sdh
and, there should be no reservation
For example: If we run the following script against a device, we can verify if SCSI-3 PR type is supported or not. For DB2 pureScale, we need support for PR Type = 5 and 7.
node02:~/bin # ./validatescsipr /dev/sdh 5 ============================================================= disk device=/dev/sdh key=b4a377d435a980f3 prtype=5 ============================================================= Check the state of the disk ............. Success Check if registration succeeds .......... Success Check if reservation succeeds ........... Success Check if release succeeds ............... Success Check if registration can be removed .... Success Check the state of the disk ............. Success ============================================================= node02:~/bin # ./validatescsipr /dev/sdh 7 ============================================================= disk device=/dev/sdh key=b4a377d435a980f3 prtype=7 ============================================================= Check the state of the disk ............. Success Check if registration succeeds .......... Success Check if reservation succeeds ........... Success Check if release succeeds ............... Success Check if registration can be removed .... Success Check the state of the disk ............. Success =============================================================
In the test environment, I am just using a simple SATA drive in a laptop and it is the driver magic that allows a SCSI-3 PR on this drive in a VM.
The iscsitarget and iscsitarget-kmp-default versions are:
iscsitarget-1.4.20-0.14.1.x86_64.rpm iscsitarget-kmp-default-1.4.20_3.0.13_0.27-0.14.1.x86_64.rpm
The script can be downloaded from this link and it is reproduced here.
#!/bin/bash
# Vikram Khatri (vikram.khatri@us.ibm.com)
# Check SCSI-3 PR Registration and Reservation
# # ./validatePR reserve /dev/sde 6
# # ./validatePR release /dev/sde 6
# Type Description
# 0 Obsolete
# 1 Write exclusive
# 2 Obsolete
# 3 Exclusive Access
# 4 Obsolete
# 5 Write Exclusive, registrants only
# 6 Exclusive Access, registrants only
# 7 Write Exclusive, all registrants
# 8 Exclusive Access, all registrants
# 9-15 Obsolete
if [ "$#" != "2" ]; then
echo "Usage: $0 diskdevice scsiprtype" 1>&2
echo "Usage: $0 /dev/sdg 7" 1>&2
exit 1
fi
ddevice=$1
sprtype=$2
if [ ! -f /usr/bin/sg_persist ] ; then
echo "Cannot locate sg_persist. Install sg3utils. Exiting..." 1>&2
exit 1
fi
if [ -f /usr/sbin/rsct/bin/lsnodeid ] ; then
key=`/usr/sbin/rsct/bin/lsnodeid`
else
key=$(gethostip -x $(uname -n))
fi
echo "============================================================="
echo "disk device=${ddevice} key=${key} prtype=${sprtype}"
echo "============================================================="
echo -n "Check the state of the disk ............. "
sg_persist -d ${ddevice} --in --read-keys > /tmp/sgp.txt
if [ $? -eq 0 ] ; then
echo " Success"
KEYS=$(sg_persist -i -k -d $ddevice | grep -E "^[\t ]*0x.*" | sed -e 's/^ *//')
for ik in $KEYS
do
sg_persist -d ${ddevice} --out -C -K $ik > /dev/null
done
else
echo " Failure RC=$?"
echo "============================================================="
cat < /tmp/sgp.txt
echo "============================================================="
exit 1
fi
echo -n "Check if registration succeeds .......... "
sg_persist -d ${ddevice} --out --register --param-sark=${key} > /tmp/sgp.txt
if [ $? -eq 0 ] ; then
echo " Success"
echo -n "Check if reservation succeeds ........... "
sg_persist -d ${ddevice} --out --reserve --param-rk=${key} --prout-type=${sprtype} > /tmp/sgp.txt
if [ $? -eq 0 ] ; then
echo " Success"
echo -n "Check if release succeeds ............... "
sg_persist -d ${ddevice} --out --release --param-rk=${key} --prout-type=${sprtype} > /tmp/sgp.txt
if [ $? -eq 0 ] ; then
echo " Success"
echo -n "Check if registration can be removed .... "
sg_persist -d ${ddevice} --out --register --param-rk=${key} --param-sark=0 > /tmp/sgp.txt
if [ $? -eq 0 ] ; then
echo " Success"
else
echo " Failure RC=$?"
echo "============================================================="
cat < /tmp/sgp.txt
echo "============================================================="
fi
else
echo " Failure RC=$?"
echo "============================================================="
cat < /tmp/sgp.txt
echo "============================================================="
fi
else
echo " Failure RC=$?"
echo "============================================================="
cat < /tmp/sgp.txt
echo "============================================================="
fi
else
echo " Failure RC=$?"
echo "============================================================="
cat < /tmp/sgp.txt
echo "============================================================="
fi
echo -n "Check the state of the disk ............. "
sg_persist -d ${ddevice} --in --read-keys > /tmp/sgp.txt
if [ $? -eq 0 ] ; then
echo " Success"
else
echo " Failure RC=$?"
echo "============================================================="
cat < /tmp/sgp.txt
fi
echo "============================================================="
Finally, RSCT command tb_break should be checked against the disk which is going to be used as a RSCT tie-breaker disk. The tie-breaker disk is reserved using PR Type = 5.
# WWID=`/lib/udev/scsi_id -g -u /dev/sdh`
# echo $WWID
1494554000000000033343536373839303132000000000000
# /usr/sbin/rsct/bin/tb_break -l -t SCSIPR WWID=$WWID
Initializing SCSIPR tie-breaker (WWID=1494554000000000033343536373839303132000000000000)
SCSIPR_init Entered
Command(/usr/sbin/rsct/bin/lsnodeid) returns exitCode=0
SCSIPR_PRKEY: lsnodeid(rc=0): 0xb4a377d435a980f3
Command(/usr/bin/sg_persist -d /dev/sg7 --out --register-ignore --param-alltgpt --param-aptpl
--param-sark=0xb4a377d435a980f3) returns exitCode=0
SCSIPR_register: '/usr/bin/sg_persist -d /dev/sg7 --out --register-ignore --param-alltgpt --param-aptpl
--param-sark=0xb4a377d435a980f3' failed
Command(/usr/bin/sg_persist -d /dev/sg7 --out --release --param-rk=0xb4a377d435a980f3 --prout-type=5)
returns exitCode=0
SCSIPR_release: device=/dev/sg7, prkey=0xb4a377d435a980f3, result=0
SCSIPR_init: Leaving registered /dev/sg7, prkey=0xb4a377d435a980f3
Reserving tie-breaker (WWID=1494554000000000033343536373839303132000000000000)
Command(/usr/bin/sg_persist -d /dev/sg7 --out --reserve --param-rk=0xb4a377d435a980f3
--prout-type=5) returns exitCode=0
SCSIPR_reserve: dev=/dev/sg7, prkey=0xb4a377d435a980f3, result=0, status=0
tb_reserve status GRANTED(0) (errno=0)
After running the above command, check the reservation on the /dev/sdh disk.
# sg_persist -d /dev/sdh -i -k
IET VIRTUAL-DISK 0
Peripheral device type: disk
PR generation=0x21, 1 registered reservation key follows:
0xb4a377d435a980f3
After locking the disk, run the following command to unlock the disk /dev/sdh.
# /usr/sbin/rsct/bin/tb_break -u -t SCSIPR WWID=$WWID sg_persist commands: To query all registrant keys for given device #sg_persist -i -k -d /dev/sdd To query all reservations for given device #sg_persist -i -r -d /dev/sdd To register #sg_persist -o -G -S 0x0007 /dev/sdd To clear all registrants #sg_persist -C -K 0x0007 -o -d /dev/sdd To reserve #sg_persist -o -R -K 0x0007 -T 5 -d /dev/sdd To release #sg_persist -o -L -K 0x0007 -T 5 -d /dev/sdd