VMware vSphere 6.x: An error occurred while consolidating disks: msg.snapshot.error-DISKLOCKED -SOLVED

Problem:

Unable to Power on Virtual Machine. It gives below Symptoms.

1. While attempting to power on giving below error.

"Power On virtual machine:Failed to lock the file

See the error stack for details on the cause of this problem.

Time: 10/9/2020 6:37:25 AM

Target: MYVM001

vCenter Server: MYVCENTER.MYDOMAIN.COM

Error Stack

An error was received from the ESX host while powering on VM MYVM001.

Failed to start the virtual machine.

Cannot open the disk '/vmfs/volumes/594d0000-d290000-500-00000000002/MYVM001/MYVM001-000002.vmdk' or one of the snapshot disks it depends on. 

Failed to lock the file"

2. Virtual Machine showing warning " Consolidation Needed". But unable to perform snapshot consolidation. It gives below error.

VMware vSphere 6.x: An error occurred while consolidating disks: msg.snapshot.error-DISKLOCKED

3. /var/log/vmkernal.log shows below error.

 Failed to open file 'MYVM001-delta.vmdk'; Requested flags 0x4008, world: 1197120 [vmx-vthread-8], (Existing flags 0x4008, world: 43796 [unknown]): Busy

4. Command "vmkfstools -e vmfs/volumes/00000003-000230-000b-0123211000/VM001-000001.vmdk" shows below output.

Failed to open disk link /vmfs/volumes/5b4d1233-3e1234-1f5b-0123211000/VM001-000001.vmdk :Failed to lock the file (16392)Disk chain is not consistent : Failed to lock the file (16392) 

5. virtual machine disk descriptor file shows normal and CID chain is not broken.

Reason:

The VMDK or Delta VMDK is locked by one of the esxi in the cluster. Even if we move the VM in the particular Host, VM will fail to power on or consolidate disk.

To identify which disk is locked, refer the /var/log/vmkernal.log

Solution:

Identify the ESXI host which is currently have lock on the Delta vmdk or VMDK with below command. 

[root@I-MY-ESXI-01:/vmfs/volumes/152525252-26252526122-ED3455555555/MYVM001] vmfsfilelockinfo -p MYVM001-000001-delta.vmdk -v 172.11.13.3 -u administrator@vsphere.local

### -v  <vcenter IP>

### -u  <vCenter credential>

Output:

vmfsfilelockinfo Version 1.0

Looking for lock owners on "MYVM001-000001-delta.vmdk"

"MYVM001-000001-delta.vmdk" is locked in Exclusive mode by host having mac address ['00:20:b2:01:10:01']

-----------------------------------------------------------------

Found 2 ESX hosts from Virtual Center Server.

------------------------------------------------------------------

Searching on Host MY-ESXI-01.MYCOM.MYDOMAIN.COM

Searching on Host MY-ESXI-02.MYCOM.MYDOMAIN.COM

    MAC Address : 00:20:b2:01:10:01

Host owning the lock on the vmdk is MY-ESXI-02.MYCOM.MYDOMAIN.COM, lockMode : Exclusive

Total time taken : 0.38 seconds.

   Above output saying, disk is locked by MY-ESXI-02.MYCOM.MYDOMAIN.COM with MAC Address 00:20:b2:01:10:01.

Steps to Clear ESXi Lock:

1. Put the host in maintenance.

2. reboot the host.

Lock will be cleared during ESXi reboot and we will be able to power on or consolidate snapshot of VM.