Linux LVM with Veritas DMP as multipathing

"dmp_native_support" parameter is required to turn on in order for the DMP to provide Linux LVM VG multipathing.

# vxdmpadm settune dmp_native_support=on
# vxdmpadm gettune  |grep support
dmp_native_support                       on              off
# lvmdiskscan |grep emc |grep LVM
  /dev/vx/dmp/emc_clariion0_1000   [       15.00 GB] LVM physical volume
  /dev/vx/dmp/emc0_05ff            [        1.01 GB] LVM physical volume
  /dev/vx/dmp/emc0_05fe            [        1.01 GB] LVM physical volume
  /dev/vx/dmp/emc0_05fd            [        1.01 GB] LVM physical volu

Linux Storage Foundatione Oracle RAC diskgroup and volume failure.

When all paths are disconnected/reconnected to SF Oracle RAC cluster node, some strange behaviour might occur. The vmax_sfracdg diskgroup appeared to be stale,  some disks like emc_0604 was showing 2 entries in the diskgroup. The diskgroup failed to deport and the volume denied to be stopped. The volume was still active in the kernel as explained in Symantec site. Reboot system is the only solution.


# vxdisk list -o alldgs |grep vmax_sfrac
emc0_0604    auto:cdsdisk    emcpoweral   vmax_sfracdg online shared
emc0_0604    auto:cdsdisk    -            (vmax_sfracdg) online shared
emc0_0605    auto:cdsdisk    emcpowerak   vmax_sfracdg online shared
emc0_0606    auto:cdsdisk    emcpoweraj   vmax_sfracdg online shared
emc0_0606    auto:cdsdisk    -            (vmax_sfracdg) online shared
emc0_0608    auto:cdsdisk    emcpowerah   vmax_sfracdg online shared

# vxdg deport vmax_sfracdg
VxVM vxdg ERROR V-5-1-584 Disk group vmax_sfracdg: Some volumes in the disk group are in use

# vxvol -g vmax_sfracdg stopall
VxVM vxvol ERROR V-5-1-1220 Volume sfracvol1 is currently open or mounted
VxVM vxvol ERROR V-5-1-1220 Volume sfracvol2 is currently open or mounted
VxVM vxvol ERROR V-5-1-1220 Volume sfracvol3 is currently open or mounted
VxVM vxvol ERROR V-5-1-1220 Volume sfracvol4 is currently open or mounted
VxVM vxvol ERROR V-5-1-1220 Volume sfracvol5 is currently open or mounted

 #vxprint -g vmax_sfracdg -m sfracvol1 |grep devopen
        devopen=on

VCS NOTICE V-16-20011-1021 CFSMount error in Storage Foundation Oracle RAC

 If CFSMount is unable to mount the resources in the service group, it might be the previous mounting is still in stale state during the previous failure. Solution is to remove the stale mounting and online the resouce again.


# df -k |grep error
df: `/vmax_sfracfs1': Input/output error
df: `/vmax_sfracfs3': Input/output error
df: `/vmax_sfracfs5': Input/output error


/var/VRTSvcs/log/CFSMount_A.log
sgelxha196:/var/VRTSvcs/log # less CFSMount_A.log
2011/09/28 16:40:30 VCS NOTICE V-16-20011-1021 CFSMount:vmax_sfrac3:monitor:File System Disabled: MountPoint: /vmax_sfracfs3
2011/09/28 16:40:30 VCS NOTICE V-16-20011-1021 CFSMount:vmax_sfrac5:monitor:File System Disabled: MountPoint: /vmax_sfracfs5
2011/09/28 16:40:30 VCS NOTICE V-16-20011-1021 CFSMount:vmax_sfrac1:monitor:File System Disabled: MountPoint: /vmax_sfracfs1

Solution:
# umount -f /vmax_sfracfs1
# umount -f /vmax_sfracfs3
# umount -f /vmax_sfracfs5

# hagrp -online vmax_sfrac -sys host1

Update VXVM&ASL/APM packages on Linux

Command to update VxVM ASL/APM.

 # ls -l
total 19968
-rw-r--r-- 1 root root   158990 Sep 21 10:50 VRTSaslapm-5.0.000.000-internal_SLES11.x86_64.rpm
-rw-r--r-- 1 root root 20258631 Sep 21 10:50 VRTSvxvm-5.0.000.000-internal_SLES11.x86_64.rpm

 # rpm -Uvh --force *.rpm --nodeps
Preparing...                ########################################### [100%]
stopping vxconfigd
   1:VRTSvxvm               ########################################### [ 50%]
Installing file /etc/init.d/vxvm-boot
vxvm-boot                 0:off  1:off  2:on   3:on   4:on   5:on   6:off
creating VxVM device nodes under /dev
Installing keys for APMs
   2:VRTSaslapm             ########################################### [100%]
Installing keys for APMs
#shutdown -r now

SF Oracle RAC (SFRAC) Service Group go into partial state after paths failure

When there are paths failure on SFRAC cluster node1 causing the node2 and/or node3 Service Group goes into partial state. This is due to the default disk detach policy set as global in the share diskgroup. "vxdg" can be used to view the setting and "vxedit" can be used to change the setting. Refer to What is the disk detach policy for shared disk groups and how can it be changed? for more detail.

#vxdg list sfracdg

#vxedit -g sfracdg set diskdetpolicy=local sfracdg

# hastatus -sum
-- SYSTEM STATE
-- System               State                Frozen
A  node1           RUNNING              0
A  node2           RUNNING              0

-- GROUP STATE
-- Group           System               Probed     AutoDisabled    State
B  cvm             node1           Y          N   ONLINE
B  cvm             node2           Y          N   ONLINE
B  sfrac           node1           Y          N   PARTIAL
B  sfrac           node2           Y          N   PARTIAL