In db2diag, you may see this message – No product license found.
Sample output:
2017-05-19-13.00.58.511626-240 E2575E1007 LEVEL: Severe PID : 35363 TID : 70366589481392 PROC : db2start INSTANCE: purf01 NODE : 000 HOSTNAME: va33dlvudb001 FUNCTION: DB2 UDB, high avail services, sqlhaExecuteCommandSQO, probe:1062 DATA #1 : String, 13 bytes va33dlvudb003 DATA #2 : String, 26 bytes /usr/sbin/rsct/bin/samlicm DATA #3 : String, 2 bytes -s DATA #4 : String, 4 bytes root DATA #5 : String, 6 bytes purf01 DATA #6 : String, 6 bytes purf01 DATA #7 : unsigned integer, 4 bytes 1055 DATA #8 : Boolean, 1 bytes false DATA #9 : SQLHA Remote Command Output, PD_TYPE_SQLHA_COMMAND_RESPONSE, 3508 bytes commandResponse->callRC: 0x00000000 commandResponse->output: No product license found. DATA #10: Hex integer, 4 bytes 0x00000000 DATA #11: Hex integer, 4 bytes 0x00000001 CALLSTCK: (Static functions may not be resolved correctly, as they are resolved to the nearest symbol) [0] 0x0000000000000000 ?unknown + 0x0 [1] 0x0000000000000000 ?unknown + 0x0 2017-05-19-13.00.58.513621-240 I3583E741 LEVEL: Error PID : 35363 TID : 70366735337216 PROC : db2start INSTANCE: purf01 NODE : 000 HOSTNAME: va33dlvudb001 FUNCTION: DB2 UDB, high avail services, sqlhaGetLicenseStatusFromEachHost, probe:24843 MESSAGE : Problem running command on the host. DATA #1 : SQLHA Remote Command Set, PD_TYPE_SQLHA_COMMAND_SET, 292120 bytes commandSet->numCommands: 4 commandSet->options: NONE commandSet->previousDb2RshCmd: DATA #2 : SQLHA Remote Command Output, PD_TYPE_SQLHA_COMMAND_RESPONSE, 8 bytes commandResponse->callRC: 0x0D9C36A0 commandResponse->output: NOT_POPULATED DATA #3 : unsigned integer, 8 bytes 3 DATA #4 : String, 27 bytes node03.zinox.com
From the above output, it looks that node03.zinox.com has the product license issue.
Fix Problem
Go to /var/opt/sam/lic
# cd /var/opt/sam/lic # ls -l -rw-r----- 1 root root 86 May 8 14:02 nodelock
If permission shows 640, change it 644
# chown 644 nodelock
Somehow, in the process of locking or unlocking this file, RSCT forgets to change the permission back. Most probably, this issue could be result of time difference between hosts but I am not sure. I will ask RSCT development on this.