ESXi 6 Update error - No Space left on device /locker

VMware ESXi 6 Upgrade Error ‘cannot execute upgrade script on host’


Error


While Upgrading VMware ESXi 5.5 to VMware ESXi 6.0 using update manager, you may receive below error.
Cannot execute upgrade script on host.
The VFAT filesystem mpx.vmhba32:C0:T0:L0:8 (UUID 5537f914-7d0da537-db5e-0025b503a03f) is full.


If you try to stage the updates, you may receive below error.
“The host returns exsupdate error code: 15. The package manager transaction is not successful.”



To get more information on error, Login to ESXi Host using SSH or local shell.
Once you logged in to ESXI Shell run below command to see recent ESXi Update logs.
#more /var/log/esxupdate.log
………………………………..
2016-01-06T09:55:22Z esxupdate: HostImage: INFO: Installer <class 'vmware.esximage.Installer.LockerInstaller.LockerInstaller'> was not initiated - reason: Failed to create empty Database directory: [Errno 28] No space left on device: '/locker/packages/var/db/locker'
……………………………………………………………..
I had above error message in my exsupdate.log file.

Resolution


Make sure you are still logged in to ESXi SSH or shell console, where ESXi update is failing.
  1. Get the ID of /locker file system in ESXi.
# cd /locker/
Then your command prompt looks like below
[root@ESXiHostname:/vmfs/volumes/553968ce-26e5e3dc-03fa-0025b503a001] #
“553968ce-26e5e3dc-03fa-0025b503a001” this is the ID of /locker file system.
Replace /locaker ID with your ESXi Host /locaker ID.

  1. Check disk utilization of /locker file system
#df -h | grep 553968ce-26e5e3dc-03fa-0025b503a001
vfat       285.8M 285.8M      8.0K 100% /vmfs/volumes/553968ce-26e5e3dc-03fa-0025b503a001


In above output you can see file system total size is 285.8 MB and all 100% space is utilized.
Even if you see few MB free space, upgrade will not work, so find all big files in /locker and remove it.
  1. Check disk space used by Directories and files.
#cd /locker/packages/
In packages directory you will have <version> dir with floppies & Vmtools directories and files inside it.
I had directory with name 6.0.0
  1. Move the version Directory to some other datastores on same ESXi Host. Just to keep it there until we finish upgrade process, then we can delete it.
#mv 6.0.0 /vmfs/volumes/<your-vmfs-datastore>/
e.g. #mv 6.0.0 /vmfs/volumes/dev-DS01/

  1. Move dump file

#mv /locker/var/core/*zdump* /vmfs/volumes/<your-vmfs-datastore>/
e.g.  #mv /locker/var/core/*zdump* /vmfs/volumes/dev-DS01/


  1. Check Free space

#df -h | grep 553968ce-26e5e3dc-03fa-0025b503a00
vfat       285.8M  48.0K    285.8M   0% /vmfs/volumes/553968ce-26e5e3dc-03fa-0025b503a001

Now we can see only 48K is being used on /locker.

ESXi Upgrade need most of the space on /locker, if you have any other big file on /locker find it and delete it.

You can use du command to check file/directory size.

#cd /locker
#du <filename>

Now go ahead and start the ESXi Upgrade process using update manager.

Below is the image with all commands which I used to fix this issue.



For similar errors like ‘Cannot execute upgrade script on host’ you can try below work arounds also.
  1. Reboot ESXi Hosts & then try to upgrade ESXi
  2. If still failed, stage the patches first and then Remediate

-
vPRH

3 comments: