DB2 10.1 pureScale requires an optional but highly recommended SCSI-3 PR capable storage.

DB2 10.1 pureScale requires a mandatory tie breaker disk which should be SCSI-3 PR capable. Optionally, you can configure an IP address to be a tie-breaker disk but this is not documented and can be configured using db2cluster command such as:

db2cluster -cm -set -tiebreaker -ip 192.168.142.2

However, the SCSI-3 PR capability regardless of the storage comes from the driver. This will be evident from the following:

On SLES 11 SP1, if we install iscsitarget and iscsitarget-kmp-default packages and use iscsitarget to do the poor man’s SAN using the disks from one server to the another, we do not get SCSI-3 capable storage.

You can check this through sg_persist command.

For example:

# sg_persist -d /dev/sdg --out --register --param-rk=0 --param-sark=0x00006d0000000001

The output may show such as

IET    VIRTUAL_DISK        0
Peripheral device type: disk
PR Out:, command not supported

However, on SLES 11 SP2, the iscsitarget driver has support for the SCSI-3 PR.

Step-1: Run the command to check the state of the disk:

# sg_persist -i -k /dev/sdh
  IET       VIRTUAL-DISK      0   
  Peripheral device type: disk
  PR generation=0xf, 1 registered reservation key follows:
    0xb4a377d435a980f3

If the above command shows that there are existing keys registered / reserved, clear those keys first using the following command before going to the next step. For example, key 0xb4a377d435a980f3 is the registered reservation on the disk.

# sg_persist -d /dev/sdh -o -C -K 0xb4a377d435a980f3 
  IET       VIRTUAL-DISK      0   
  Peripheral device type: disk

Again check the state of the disk by using

# sg_persist -i -k /dev/sdh

and, there should be no reservation

For example: If we run the following script against a device, we can verify if SCSI-3 PR type is supported or not. For DB2 pureScale, we need support for PR Type = 5 and 7.

node02:~/bin # ./validatescsipr /dev/sdh 5
=============================================================
disk device=/dev/sdh key=b4a377d435a980f3 prtype=5
=============================================================
Check the state of the disk .............  Success
Check if registration succeeds ..........  Success
Check if reservation succeeds ...........  Success
Check if release succeeds ...............  Success
Check if registration can be removed ....  Success
Check the state of the disk .............  Success
=============================================================

node02:~/bin # ./validatescsipr /dev/sdh 7
=============================================================
disk device=/dev/sdh key=b4a377d435a980f3 prtype=7
=============================================================
Check the state of the disk .............  Success
Check if registration succeeds ..........  Success
Check if reservation succeeds ...........  Success
Check if release succeeds ...............  Success
Check if registration can be removed ....  Success
Check the state of the disk .............  Success
=============================================================

In the test environment, I am just using a simple SATA drive in a laptop and it is the driver magic that allows a SCSI-3 PR on this drive in a VM.

The iscsitarget and iscsitarget-kmp-default versions are:

iscsitarget-1.4.20-0.14.1.x86_64.rpm
iscsitarget-kmp-default-1.4.20_3.0.13_0.27-0.14.1.x86_64.rpm

The script can be downloaded from this link and it is reproduced here.

#!/bin/bash
# Vikram Khatri (vikram [dot] khatri [at] us [dot] ibm [dot] com)
# Check SCSI-3 PR Registration and Reservation

# # ./validatePR reserve /dev/sde 6
# # ./validatePR release /dev/sde 6
# Type       Description
# 0          Obsolete
# 1          Write exclusive
# 2          Obsolete
# 3          Exclusive Access
# 4          Obsolete
# 5          Write Exclusive, registrants only
# 6          Exclusive Access, registrants only
# 7          Write Exclusive, all registrants
# 8          Exclusive Access, all registrants
# 9-15       Obsolete

if [ "$#" != "2" ]; then
   echo "Usage: $0 diskdevice scsiprtype"  1>&2
   echo "Usage: $0 /dev/sdg 7"  1>&2
   exit 1
fi

ddevice=$1
sprtype=$2

if [ ! -f /usr/bin/sg_persist ] ; then
   echo "Cannot locate sg_persist. Install sg3utils. Exiting..." 1>&2 
   exit 1
fi

if [ -f /usr/sbin/rsct/bin/lsnodeid ] ; then
   key=`/usr/sbin/rsct/bin/lsnodeid`
else
   key=$(gethostip -x $(uname -n))
fi
echo "============================================================="
echo "disk device=${ddevice} key=${key} prtype=${sprtype}"
echo "============================================================="
echo -n "Check the state of the disk ............. "
sg_persist -d ${ddevice} --in --read-keys > /tmp/sgp.txt
if [ $? -eq 0 ] ; then
   echo " Success"
   KEYS=$(sg_persist -i -k -d $ddevice | grep -E "^[\t ]*0x.*" | sed -e 's/^ *//')
   for ik in $KEYS
   do
      sg_persist -d ${ddevice} --out -C -K $ik > /dev/null
   done
else
   echo " Failure RC=$?"
   echo "============================================================="
   cat < /tmp/sgp.txt
   echo "============================================================="
   exit 1
fi   
echo -n "Check if registration succeeds .......... "
sg_persist -d ${ddevice} --out --register --param-sark=${key} > /tmp/sgp.txt
if [ $? -eq 0 ] ; then
    echo " Success"
    echo -n "Check if reservation succeeds ........... "
    sg_persist -d ${ddevice} --out --reserve --param-rk=${key} --prout-type=${sprtype} > /tmp/sgp.txt
    if [ $? -eq 0 ] ; then
        echo " Success"
        echo -n "Check if release succeeds ............... "
        sg_persist -d ${ddevice} --out --release --param-rk=${key} --prout-type=${sprtype} > /tmp/sgp.txt
        if [ $? -eq 0 ] ; then
            echo " Success"
            echo -n "Check if registration can be removed .... "
            sg_persist -d ${ddevice} --out --register --param-rk=${key} --param-sark=0 > /tmp/sgp.txt
            if [ $? -eq 0 ] ; then
                echo " Success"
            else
                echo " Failure RC=$?"
                echo "============================================================="
                cat < /tmp/sgp.txt
                echo "============================================================="
            fi
        else
            echo " Failure RC=$?"
            echo "============================================================="
            cat < /tmp/sgp.txt
            echo "============================================================="
        fi
    else
        echo " Failure RC=$?"
        echo "============================================================="
        cat < /tmp/sgp.txt
        echo "============================================================="
    fi
else
    echo " Failure RC=$?"
    echo "============================================================="
    cat < /tmp/sgp.txt
    echo "============================================================="
fi
echo -n "Check the state of the disk ............. "
sg_persist -d ${ddevice} --in --read-keys > /tmp/sgp.txt
if [ $? -eq 0 ] ; then
   echo " Success"
else
   echo " Failure RC=$?"
   echo "============================================================="
   cat < /tmp/sgp.txt
fi   
echo "============================================================="

Finally, RSCT command tb_break should be checked against the disk which is going to be used as a RSCT tie-breaker disk. The tie-breaker disk is reserved using PR Type = 5.

# WWID=`/lib/udev/scsi_id -g -u /dev/sdh`
# echo $WWID
1494554000000000033343536373839303132000000000000
# /usr/sbin/rsct/bin/tb_break -l -t SCSIPR WWID=$WWID
Initializing SCSIPR tie-breaker (WWID=1494554000000000033343536373839303132000000000000)
SCSIPR_init Entered
Command(/usr/sbin/rsct/bin/lsnodeid) returns exitCode=0
SCSIPR_PRKEY: lsnodeid(rc=0): 0xb4a377d435a980f3
Command(/usr/bin/sg_persist -d /dev/sg7 --out --register-ignore --param-alltgpt --param-aptpl   
   --param-sark=0xb4a377d435a980f3) returns exitCode=0
SCSIPR_register: '/usr/bin/sg_persist -d /dev/sg7 --out --register-ignore --param-alltgpt --param-aptpl   
   --param-sark=0xb4a377d435a980f3' failed
Command(/usr/bin/sg_persist -d /dev/sg7 --out --release --param-rk=0xb4a377d435a980f3 --prout-type=5) 
    returns exitCode=0
 SCSIPR_release: device=/dev/sg7, prkey=0xb4a377d435a980f3, result=0
 SCSIPR_init: Leaving registered /dev/sg7, prkey=0xb4a377d435a980f3
Reserving tie-breaker (WWID=1494554000000000033343536373839303132000000000000)
Command(/usr/bin/sg_persist -d /dev/sg7 --out --reserve --param-rk=0xb4a377d435a980f3 
  --prout-type=5) returns exitCode=0
 SCSIPR_reserve: dev=/dev/sg7, prkey=0xb4a377d435a980f3, result=0, status=0
tb_reserve status GRANTED(0) (errno=0)

After running the above command, check the reservation on the /dev/sdh disk.

# sg_persist -d /dev/sdh -i -k
  IET       VIRTUAL-DISK      0   
  Peripheral device type: disk
  PR generation=0x21, 1 registered reservation key follows:
    0xb4a377d435a980f3

After locking the disk, run the following command to unlock the disk /dev/sdh.

# /usr/sbin/rsct/bin/tb_break -u -t SCSIPR WWID=$WWID

sg_persist commands:

To query all registrant keys for given device
#sg_persist -i -k -d /dev/sdd

To query all reservations for given device
#sg_persist -i -r -d /dev/sdd 

To register
#sg_persist -o -G -S 0x0007 /dev/sdd

To clear all registrants
#sg_persist -C -K 0x0007 -o -d /dev/sdd 

To reserve
#sg_persist -o -R -K 0x0007 -T 5 -d /dev/sdd

To release
#sg_persist -o -L -K 0x0007 -T 5 -d /dev/sdd